Jev Routing Function Testing

Sep 20

Jev was tested on core routing functions, demonstrating accuracy comparable to or exceeding current LLMs, with significantly improved speed and reduced cost.

Pattern↑ Jev AI decision models

Congrats @typesafeai team... tested Jev on some @HeyHelixAI core routing functions (for example "Is this a flight / hotel / restaurant / experience request?") against 100s of evals. Early test results: - Accuracy on par with, or slightly ahead of, the (already small, fast, cheap) LLMs we use today, on real user messages the models had never seen ✅ - Roughly 4x faster: about a quarter of a second per decision, versus 1 to 1.7 seconds (note some of this was network latency from Europe - US) ⚡ - A fraction of the cost per call 💷 - Confidence that means something: when it said it was confident (0.8+), it was right 96 to 100% of the time 🎯 - Matching remembered traveller preferences to hotels ("does this hotel have what I like?"): it found 97.6% of real matches, with zero false positives 🏨 Headline image from typesafe.ai - go check it out

Original post