CHOOSESearch & retrieval
Jev Decision Model Benchmark
WilliamZhu@allwefantasy
Jev, a decision model, was benchmarked against a recall-judgment LLM using Apple 10-K filings, achieving tied accuracy with significantly lower latency.
We benchmarked Jev — TypeSafe's first "System One" decision model — against the recall-judgment LLM we run in production. Corpus: 10 years of Apple 10-K filings, 79 question-doc pairs. Result: accuracy tied at 92.2%, latency 1.0s vs 11.3s.
Related builds
Vectorless RAG with Jev
This project introduces a novel approach to Retrieval Augmented Generation (RAG) that replaces traditional embedding methods with a structured, table-of-contents-based navigation system, significantly improving speed and reducing costs by offloading non-generative tasks to Jev.
