Typed Decisions vs JSON Mode: What's Actually Different

JSON mode, function calling, and constrained decoding are all ways of making a chat LLM's free-text output conform to a shape — they're guardrails wrapped around an autoregressive text generator. Jev, TypeSafe AI's System One model, doesn't generate text at all: it's a non-autoregressive parallel sampler whose output is natively a typed value, not a string that happens to parse as JSON. That architectural difference is the whole story here.

What does JSON mode actually guarantee?

JSON mode constrains a chat LLM's decoding so each generated token keeps the output on a path toward valid JSON syntax — brackets balance, strings are quoted, keys match a schema. It's a real engineering improvement over asking a model to "please respond in JSON" in a prompt. But the model is still autoregressive: it's generating a token sequence, and the constraint operates on syntax, not on whether the field values it chooses are sensible or accurate.

What does function calling add on top of that?

Function calling (or "tool use") has the LLM choose which function to invoke and generate arguments matching that function's parameter schema — still via constrained token generation, just with the schema tied to a specific tool definition instead of a generic JSON shape. It's the closest LLM-native analog to what Jev does, and it's the right comparison point: both let you define a schema up front and get output matching it.

So what's different about Jev's approach?

The generation mechanism. Function calling and JSON mode are constraints layered on an autoregressive transformer — the model still predicts token N+1 conditioned on tokens 1..N, just with the vocabulary restricted at each step to keep the output on-schema. Jev uses what TypeSafe AI describes as a parallel sampler on a non-transformer architecture that produces all outputs for a query in a single pass, with no token-by-token string generation involved at any point. See non-autoregressive models for how that changes latency.

	JSON mode / function calling	Constrained decoding (general)	Jev (typed decision)
Underlying generation	Autoregressive, token-by-token	Autoregressive, token-by-token	Parallel sampler, non-autoregressive
Guarantees	Output parses as valid JSON	Output matches a grammar/schema	Output matches schema (0% error rate, vendor-reported)
Content correctness	Not guaranteed	Not guaranteed	Not guaranteed (67.8% accuracy on TypeSafe's benchmark)
Typical latency	Full LLM generation time	Full LLM generation time	70–500ms (vendor-reported)
Question shape	Arbitrary schema	Arbitrary grammar	Choice / Score / Noul, cardinality ≤255

Is constrained decoding the same idea as Jev's typed output?

They share a goal (guaranteed schema conformance) but not a mechanism. Constrained decoding — grammar-based sampling, regex-constrained generation, and similar techniques — masks the LLM's token probabilities at each generation step so only schema-valid tokens can be chosen. It's still fundamentally sequential and still runs at LLM-scale latency. Jev's schema conformance comes from the architecture itself, which is why TypeSafe AI's latency numbers (70–500ms) are an order of magnitude or more below typical constrained-decoding LLM calls.

Does "guaranteed schema" mean "guaranteed correct"?

No, for any of these approaches. A JSON-mode LLM can return valid JSON with a wrong classification in it. Jev can return a schema-valid Choice, Score, or Noul that's also wrong — TypeSafe's own benchmark puts Jev at 67.8% accuracy, meaning roughly one in three answers is incorrect even though 100% are structurally valid. This is the central distinction to hold onto across all of these techniques: format validity and content correctness are separate properties, and none of these methods close that gap by themselves. See structured-output error rates for how to measure both.

Common pitfalls

A few mistakes come up repeatedly when comparing these approaches:

• Assuming "structured output" and "correct output" are the same claim — they aren't, for any of these approaches.
• Comparing Jev's latency to an LLM's JSON-mode latency without accounting for the fact that Jev can't do open-ended generation at all.
• Using function calling for a high-volume, latency-sensitive classification task when a purpose-built typed-decision model would be cheaper and faster.
• Expecting a self-hosted grammar constraint to match Jev's reported 70–500ms without benchmarking your own setup.