VERIFYSearch & retrieval
Jev Performance Benchmark
Keiran Haax@keiranhaax
0:44Jev demonstrates significantly faster performance compared to another model in a controlled test involving document searching and evidence evaluation.
Jev by @typesafeai was ~7× faster in this controlled side-by-side test (1× speed)
Same loop on both lanes: pick a query → fork of Omnisearch mcp → official source → read → judge the evidence
End-to-end, Jev vs Lina’s model:
• TypeSafe docs: 1.47s vs 8.96s
• Python docs: 1.28s vs 9.22s
• SQLite docs: 1.23s vs 6.86s
Both 3/3
Median total: 1.28s vs 8.96s
Median decision: 213ms vs 2,436ms
Boundary: Jev vs gpt-6-astra, fresh context, low reasoning
Prepared query choices; neither generated searches freely