That is a decision model in two sentences, from Aaron Francis replying to James McDonald on X. The rest of this article is the same distinction with the thresholds, the file names and the numbers attached.
A chat model writes the things Routines makes for you, the morning digest, the meeting notes, the draft post. That model is Claude or ChatGPT, or whatever you reach through OpenRouter. You run it on your own account, or on the AI Routines provides. Writing is the job it is good at. Four times we also asked it for a verdict: continue or stop, pass or fail, keep or drop, file here or nowhere. A fifth verdict, whether a finished output is worth a badge, was never a model call at all, only a keyword test. All five now go to a decision model that returns a calibrated probability and cannot write a sentence: four of them in v1.6.0, out now, and the folder pick earlier, in v1.5.15. The model is Jev, and it is TypeSafe AI's, not ours.
The five places we needed a verdict
- Flow edge gate: does this flow continue down this edge.
- Output double check: does this output meet its goal.
- Meeting to-do owner: is this action item yours or theirs.
- Meeting folder pick: which folder in your notes do these meeting notes go in.
- Needs-attention badge: is this output worth a badge.
Four of them work the same way. We ask a chat model the question, tell it to answer in a fixed format, and keep a second piece of code underneath to rescue the answer on the runs where it answers in some other format. That second piece is the tell: we were paying a writer to do a judge's job and then cleaning up after it. No tools, no conversation history, just the question, with the randomness turned most of the way down. The gate and the double check run at the fast tier at temperature: Some(0.0), the to-do owner at 0.1, and the folder pick a tier up at 0.2.
The fifth was never a model call. The badge is a keyword test, which is what it sounds like. The screen scans the text for "high-priority", "high priority" or "urgent", raises the badge if it finds one, and does it again every time the screen redraws.
Here is one of those four prompts, the output double check's grader:
You are an impartial reviewer. You are given the GOAL of an automated task and
the OUTPUT it produced. Judge ONLY whether the output fulfills the goal and is
internally consistent (dates, numbers, and claims do not contradict each other).
Respond with ONLY a JSON object of the exact shape {"pass": boolean, "reason":
string, "fix": string} where "reason" is one short sentence and "fix" is a
concrete instruction for what to change (empty string when pass is true). No
prose, no markdown.
The gate's older prompt told the model to answer continue when it could not tell, and so laundered every uncertain run into a clean green pass. We fixed it, and the doc comment now states the rule it keeps: "a gate that broke is never recorded as a clean verdict." The same comment names the site that still launders: "verify_output absorbs an unreadable grader result into a plain passed; flows must not." That one is live, deliberate and pinned by a test, because an unreadable verdict must never fail the run.
The five sites, one by one
Flow edge gate. A flow is where a routine triggers another routine when a condition matches. You chain the steps on a canvas, and on the line between two steps, the edge, you write the condition in your own words. The flow walks that line only when the condition holds.
Something has to read the previous step's result and say whether it holds. A chat model used to do it. It wrote a sentence about the condition, and a parser underneath tried to recover a verdict from the prose. The same condition now goes to Jev, which answers it as one probability and nothing else, and the flow walks that edge only when the probability is 0.85 or higher. Below that nothing stops, the evaluation escalates to the chat gate that ran before, and the edge's own setting still decides what an undecided check means.
Take the Linkedin post flow below. It runs AI News Radar, then Daily LinkedIn News Post, then LinkedIn Post Publisher, and the edge out of the radar carries one condition: Only run if important news for BinarCode.
For engineers: GATE_JEV_CONTINUE_MIN = 0.85. At or above it the flow continues and spends no chat call; below it, the evaluation escalates to the chat gate. Jev may never stop a flow.
The Linkedin post flow. The gate sits on the edge out of AI News Radar, and every step in this run finished.
The condition Jev now answers as a single probability. The check reads the first ~12,000 characters of the previous step's result, and the setting underneath decides what happens when it cannot tell.
Output double check. A routine in Goal mode keeps retrying until its result meets the success criteria you wrote. You type those criteria into the routine editor as a sentence. Something has to check each result against those criteria.
Until v1.6.0 a chat model did that reading: a grader prompt returned a pass or a fail with a reason, and code underneath pulled the verdict out of the reply. Jev now grades the output first, against a fixed three level scale written in our own code. A score of 0.85 or higher on that scale, measured as a fraction of the top level, delivers the run passed and spends no grader call. Everything below that still goes to the same grader, because a shortfall has to come back with a fix sentence to retry against, and a typed decision cannot write one.
The routine below runs every morning in Goal mode, and its criteria read: the output has exactly 4 sites.
For engineers: VERIFY_JEV_PASS_MIN = 0.85 on the score's normalized() value, plus VERIFY_JEV_CONFIDENCE_MIN = 0.6 when a confidence is measured. From VERIFY_JEV_CONCERN_MAX = 0.5 to 0.85 it escalates to the prose grader. Below 0.5 it escalates too, and marks a concern: that is the one band where an unreadable grader verdict is no longer absorbed into a pass.
Goal run mode. The success criteria go to the check along with the routine's name and prompt, and Jev scores that pairing before the prose grader sees it.
Meeting to-do owner. After a meeting Routines pulls out the action items and names an owner for each one. Only the ones that are yours should become tasks in your list. Somebody else's commitment is not your to-do.
That ownership question used to come back from a chat model as a word, with code underneath to read the word. It now comes back as a probability: 0.75 or higher keeps the item, below 0.25 drops it, and everything between the two lines falls through to the verdict the old path already reached. The keep line sits below 0.85 on purpose. A wrong to-do dies in one gesture, a dropped one is a commitment nobody sees again.
The shot below is where a kept one lands: the Tasks view, grouped by when it needs you. Under Today sits one of them, Unblock iPhone push notifications: mint the APNs key into Forge, then plug the iPhone in by USB for the device proof.
For engineers: TODO_OWNER_USER_MIN = 0.75 keeps the item, TODO_OWNER_DROP_MAX = 0.25 drops it, and between them it falls through to today's string verdict.
The Tasks view. An action item only becomes a row here after the owner filter keeps it.
Needs-attention badge. Home shows a small amber row on a routine output that needs you to look at it. It is there to pull your eye to the one result out of the morning's batch that cannot wait.
This site was never a model call. The screen scans the output text for high-priority, high priority or urgent, raises the badge on a match, and does it again every time the screen redraws. Jev is now allowed to raise the badge when it answers 0.85 or higher, and to do nothing else. On a confident no it answers nothing, the column stays NULL, and the row stays up.
The shot below is Home on a Mac that ran AI News Radar this morning. The row reads AI News Radar flagged urgent content, and the digest behind it is a news roundup that happens to quote the word urgent. That is the keyword test doing what it has always done, because a substring is all it has to work with. Suppressing that row is the named next step, and v1.6.0 did not ship it.
For engineers: ATTENTION_FLAG_MIN = 0.85 raises the badge. Everything else answers nothing, the column stays NULL, and the keyword test runs as before.
The badge on Home. A news digest that merely quotes the word urgent still collects it.
Figure 1. The gate before and after. Only a confident yes skips the chat call.
Figure 2. Every band, every constant. Nothing acts at or below 0.5, because on a two option question chance alone puts 0.5 on the winner.
Meeting folder pick. When a meeting ends, Routines files the notes in one of your note folders. It has to pick which one. Pick well and the notes sit with the project they belong to.
A chat model used to choose, and code underneath read a folder name out of its reply. Once a key is stored and there are at least two folders it could file into, Jev answers and the chat request is never built, so the expensive call never happens. A pick that clears the line files the notes in the folder it names, and anything below leaves them in the default location. Saying no is cheap here, which is why the line is 0.6 and not 0.85: a refused pick leaves the notes where they already are, while a confident wrong pick buries them inside somebody else's project folder.
Below is that band from the outside. Routines recorded, summarized and filed a Hackathon meeting on 18 September, and the footer under the notes names the file it went to: meetings/2026-09-18/1115-meet-hackathon-binar.md. That is the default location, and the one every Mac without a key gets for every meeting.
For engineers: MIN_FOLDER_PICK_CONFIDENCE = 0.6; below it the notes go to the default location. It shipped in v1.5.15, and the four bands above it are what v1.6.0 added.
The default file. This is where meeting notes land when the folder pick does not clear 0.6, and where they land on every Mac with no key at all.
All five are built the same way. Each has a named constant you can read in the source, a route for an answer Jev is not sure about, and the same no-key guarantee: with no decision key stored, the typed path does not exist and the old branch runs.
The live numbers
The live proof is four ignored probe tests, one per site. The badge's probe loops over two cases, so four probes make five calls: $0.000016, $0.000019, $0.000028, $0.000020 and $0.000019, totalling $0.000102. Our own report said $0.000083, which is the first four of those five: it counted probes, not calls.
Figure 3. Cost per call, re-derived from the token counts at $0.042 per million input tokens. Latency is not charted: nothing in this codebase times a model call.
At $0.042 per million input tokens with output free, the gate's estimated 3,250 tokens cost $0.000137 against $0.0034 for the chat verdict, 25 times cheaper; the grader's 3,560 tokens cost $0.000150 against $0.0040, 27 times. Those token counts are the finder's estimates, not measured request bodies, and the chat side is priced on the Haiku 4.5 fallback basis: this Mac runs a tier with no price row, so the app stores NULL, never a fabricated zero. At this Mac's measured volume for both sites, zero, the saving per day is exactly $0.00. Jev's published latency is 70 to 500 ms, the vendor's number and not ours.
Figure 4. The pair that made the case. The shipped keyword test is wrong in both directions; Jev is right in both.
The pair that made the badge's case ran against jev-1.13.0. The decoy: "UrgentIQ raised a 15M Series A this week. Analysts call the compliance gap it targets large, urgent and under-served." The keyword test badges it. Jev returns needs_attention 0.03 and raises nothing, because the question's own false criterion covers it: an output that "names a company or product whose name contains such a word" is not one that needs you. The second: "The landlord replied and needs your signature on the lease addendum before Friday or the renewal lapses." The keyword test says FALSE; Jev returns 0.97.
Figure 5. Both outputs through both paths, the substring test and the judge.
We got one precision wrong in review and caught it before it became a ticket. The same call also asks a how_soon score, and on the decoy that came back at 0.08 with 92 per cent of its mass on "no action is needed" and a confidence of 0.88. The 0.88 belongs to the score, never to the 0.03. A noul, the plain yes or no shape, carries no confidence field. The score is logged, never acted on: output tokens are free, and a later retune of ATTENTION_FLAG_MIN will want this Mac's own distribution.
What a System One decision model is
Jev is TypeSafe AI's System One model, and the launch post is theirs to explain. The wire contract is a POST to https://api.typesafe.ai/v1/systemone under a Bearer key, carrying state, model and a map of questions whose keys are yours. A noul answers yes or no with one probability between 0 and 1. A choice picks one of a map of named options, with a probability for each and a confidence. A score places it on an ordered rubric of at least two levels and returns a score, a distribution, a legend and a confidence.
Here it is in Postman, as small as a decision call gets. The state is what Jev looks at, one sentence in this case. The model names which one to ask. The questions map holds what you want decided, and the key sky_is_blue is a name we picked, so the answer comes back filed under it. That is the whole request. Nothing in it tells the model how to phrase a reply, because it cannot phrase anything.
Figure 6. The smallest decision call there is. The answer is a number, and the number is the whole answer.
Back came "noul": 0.9. That is the entire answer: the sky is blue with 90 per cent probability, with no sentence around it to read past and no confidence field beside it, because on a yes or no question the probability is the uncertainty. The call billed 288 input tokens, about $0.000012 at $0.042 per million, and the 22 output tokens cost nothing because TypeSafe charges only for input. Postman timed the round trip at 265 ms, which is one call from one laptop over the public internet rather than a measurement of anything.
The real requests look the same, only longer. The needs_attention half of the badge's request, with the two criteria strings elided and the state stood in for. A how_soon score rides in the same call.
{
"state": {
"routine_name": "<the routine's name>",
"output_excerpt": "<the first 800 characters of the output>"
},
"model": "jev-latest",
"questions": {
"needs_attention": {
"type": "noul",
"instructions": "Does this output tell the user about something of their own that needs action soon?",
"criteria": {
"true": "[...]",
"false": "[...] names a company or product whose name contains such a word"
}
}
}
}
This came back on a decoy we wrote to break the keyword test, usage and legend elided.
{
"model": "jev-1.13.0",
"answers": {
"needs_attention": { "type": "noul", "noul": 0.03 },
"how_soon": {
"type": "score",
"score": 0.08,
"probabilities": { "0": 0.92, "1": 0.08, "2": 0.0 },
"confidence": 0.88
}
}
}
A noul answer has no confidence field. Its only fields are its type and its probability, and the probability is the uncertainty. Jev cannot write text at all, so it replaces the parse and leaves the prose to the chat model. Laravel's AI SDK shipped the same shape on 2026-09-17 in PR laravel/ai#1010, as Boolean, Choice and Score, provider id typesafe, default model jev-latest. We mirrored its names.
The architecture: LLM generates, Jev decides, code controls
The LLM still writes. Jev returns a number. Code owns the threshold, a named constant you can read. When the number falls short, the answer routes somewhere that already existed: the chat path, or a person.
Figure 7. LLM generates, Jev decides, code controls, and the uncertain band routes to a path that was always there.
Figure 8. One decision, end to end. With no key stored, the run takes the path it always took.
Tests pin the no-key guarantee. The decide tool registers only when the app finds a decision key, and the four sites reach Jev through that same key check, so the badge's pin asserts nothing was sent and the stop happens before a request is built. This one pins the behaviour we criticised above.
#[tokio::test]
async fn with_no_key_an_unreadable_verdict_is_still_absorbed_into_passed() {
// THE pin. Today an unreadable grader verdict delivers the run marked
// "passed" with no note, and with no decision key that must not change
// by a single byte.
let (result, sends) = run_double_check(
JevVerdict::Unjudged,
&verify_long_output(),
vec![Ok("totally not json".to_string())],
)
.await;
let result = result.expect("a verdict was returned");
assert_eq!(result.status, "passed");
assert_eq!(result.content, verify_long_output());
assert_eq!(result.note, None);
assert_eq!(sends, 1);
}
How we chose where to put it
Four blind finders swept the codebase by pattern, by subsystem, by data and by cost. We merged their candidates and measured volume against the app's own SQLite database. The formula is expected win times measured volume, divided by risk, both on a 1 to 5 scale, where risk is how badly and how silently a wrong answer lands, times how much surface the change touches.
Figure 9. The inventory funnel. The per finder counts were never recorded, so the figure names the four angles and counts only what the review doc actually holds.
The raw count before dedupe was not recorded either, so we put no number on it. What survived: 18 ranked candidates, 4 implement now, 8 later, 6 never.
The two candidates the brief itself seeded, the gate and the double check, both measured zero current volume: zero of 6 flow edges carry a condition, zero of 687 outputs carry a verify status. They shipped as ranks 1 and 2 anyway, each on its own stated forward assumption. The gate's: flow runs ran 2 per day from 2026-08-08 to 2026-09-03 and then stopped, so 2 runs a day on 1 gated edge is 2 evaluations a day. Grounded in history, but an assumption, and the review labels it as one.
On measured volume the order differs: 6.9 for the badge, 3.4 for the to-do owner, 5.0 for the gate on its assumed volume, 0.78 or 17.0 for the grader, both of those on assumed volume as well. The highest measured volume of the four, 10.27 completed routine outputs a day, belonged to the badge, which neither the brief nor half the finders led with, because it was a substring test and not a model call.
The laws we banked
- Nothing acts at or below 0.5 on a two option question: chance alone puts 0.5 on the winner.
- An "unsure" option inverts calibration: it makes uncertainty an answer and takes mass from the real outcomes, and with three options chance sits at 0.33.
- Options come from code, never from the state. A choice's winning option name is logged, which makes this load bearing.
- Never gate a tool-enabled agent behind a classifier.
- A
noulhas no confidence field. The probability is the uncertainty. - Log the decision, never the state.
- Every uncertain answer needs a written route, and a transport error takes that route too.
- Keep arithmetic and hard rules in code. Jev judges meaning.
What we did not do, and what is next
Six candidates were marked never. The meeting chat intent router stays a router: law 4 is quoted from our own project memory, and failing open makes it a gate that fails safe, not a gate that stopped being one. The Slack access allowlist stays a hard rule, because a probabilistic access control is a defect by construction. Device sync conflict auto-merge stays manual: a wrong merge is silent, irreversible loss inside the user's own vault, and the conflict count is zero, so there is no calibration data either. Clipboard classifiers are out, because clipboards carry passwords and tokens. The hard rule matchers compare exact internally generated strings, several in hot loops where a network call is disqualifying. The managed wallet proxy is off limits by the brief's own rule.
Three things are next. The confident negative: the badge's judge never returns a confident no today, so one falls through to the keyword test and the bad badge shows. The decoy produced the calibration datum that was missing, and the column and the component already handle a false. We left it out of this pass, because changing decision semantics after an adversarial approval without a re-verify is how unverified behaviour ships. Thresholds tuned on our own data: every constant above is a judgment call, and the user rating column is NULL on every output row, so there is no labelled corpus yet. A pinned model version: jev-latest tracks stable and runs today; once tuned, the pin moves to jev-1.13.0, so a vendor update cannot move a threshold underneath us.
The routines decide verb and the decide tool shipped in v1.5.15, and the CLI reference documents the verb, including the TypeSafe or OpenRouter key it requires. Four of the five sites above are live in v1.6.0. The TypeSafe key card is in Settings under Connections. An OpenRouter key sits on the AI provider settings, where it already is if you use one for chat. Either way the key lives in the macOS Keychain and never syncs between Macs. Without one, nothing about the app changes. With one, the text Jev judges is sent to that provider, and the decision log keeps the verdict, not the words: never the state text, no excerpt, nothing derived from it, with a test that fails if it does.
Routines is a free download for macOS, with a CLI and its command reference. One of the decisions above is visible from the outside: the needs-attention badge, on any routine you build from the inbox triage recipe.