DETECTDevtools & code
Jev: Probabilistic Decision Model
Robot-man@RobotsTJ500
Jev is a model that provides probabilistic decisions on defined questions instead of generating text, operating in a single parallel pass with typed inputs and outputs.
TakeJev flips this — input is a block of state plus typed questions (yes/no, choice, numeric score), output is probabilities and typed answers in one parallel pass.
Pattern↑ State to typed decisions for software
Jev — a model from TypeSafe AI (@typesafeai) that never generates text. Instead of strings it returns probabilistic decisions on questions you define. Founder Diogo Almeida, co-inventor of ChatGPT at OpenAI.
What it is: a regular LLM writes tokens you parse and validate afterwards. Jev flips this — input is a block of state plus typed questions (yes/no, choice, numeric score), output is probabilities and typed answers in one parallel pass. Stateless: no session context, no prose back. Trained with RLCD — calibrated decision probabilities, not human preference. Live-API nuance: the yes/no type (noul) has no confidence field — the probability is the confidence; choice and score carry explicit confidence.
Numbers from the primary source: $0.042 per million input tokens ($42 per billion), output free; claimed latency 70–500 ms — on our real runs with 5–20k-token states we measured 0.8–1.1 s; limit 64k tokens per request; on TypeSafe's own benchmark ~68% accuracy, mid-tier LLM level, 40–400x cheaper.
Caveat: TypeSafe built those benchmarks themselves — design bias, which they acknowledge. "Cannot hallucinate" is not "cannot be wrong": a valid answer can be incorrect.
The compaction wave: a Sep 17 hype post about "instant compaction" pulled 3.7M views — score every tool call, drop the irrelevant ones from agent context. Real wrappers: Jev answers two yes/no questions per call — keep the call, keep the result verbatim.
@Teknium (cofounder of Hermes Agent, @NousResearch) ran it on a public, reproducible compaction eval: the result reduced to a rule — "delete all tool calls from history". Plain code does that free. Then a vicious cycle: each pass leaves fewer tool calls to remove, until a hard stop. And every compaction breaks the cache — 10x the input price.
@theo (t3.gg) went deeper: compaction is not a filter; this Jev sees neither the decision history nor the tool result — stupid loops follow; reasoning traces on frontier APIs are encrypted and get dropped — the model is measurably dumber after such compaction; and cache writes invalidate on history edits — deleting from the middle means rewriting the tail.
@tonysimons_: don't buy tools off headlines — show the benchmarks.
Our experience — a different use case, different numbers. We use Jev as a triage layer in code review: fast pre-screen of diffs before the manual pass. Every number script-verified:
- Diff pre-screen (5–7k-token state): 7,125 in / 107 out, ~0.9 s. Calibration 5/5 against the manual pass, zero hallucinations.
- Full-tree review: 20,273 in / 164 out — flags matched the manual review, human-confirmed.
- Fix-diff review: 12,571 in / 281 out — 38% fewer tokens, wider coverage, zero false positives.
- One flag refuted manually (model wrong), one confirmed — a human miss the model caught.
Wording matters: "does X happen" scores 0.97; framed as "is there a defect X" — 0.12. The frame "judging only by this diff" suppresses hallucinations about absent things. State size matters: 45k tokens in one block = dilution; working size is a hunk with context, 5–7k.
Our contract: flags above threshold = "open first, confirm the mechanism"; below = "no signal", not "safe". The manual pass stays policy. Jev sets the order of eyes, not the filter.
Honest limits: not an LLM replacement — useless for open-ended generation, chat, code; for repeated decisions over a known answer space. No "why" — just a probability. No real pricing page in docs — we measure in tokens.
The criticism hit specific compaction wrappers, not the typed-decisions idea. A layer of fast calibrated decisions over an agent works — we measured it. One that silently cuts agent context and breaks the cache does not.
The trend: the Decision API as its own class — typed probabilities for pocket change replace what we did with a prompt to a big model. Benchmarks before adoption: an eval you can run yourself is worth more than a million views — @Teknium, @theo, @tonysimons_ proved it.
Building in public. 🤖
Related builds
Vectorless RAG with Jev
This project introduces a novel approach to Retrieval Augmented Generation (RAG) that replaces traditional embedding methods with a structured, table-of-contents-based navigation system, significantly improving speed and reducing costs by offloading non-generative tasks to Jev.
