CHOOSEBusiness workflows
JEV for Classification Tasks
john kutay@JohnKutay
JEV is an agent harness tested in GrowthOS for classifying campaign emails by strategy, achieving 90% accuracy and running 8x faster than an LLM baseline.
TakeJEV performed poorly as a control mechanism inside our text-to-SQL agent loop because it lacked the semantic context needed to interpret internal business definitions.
We tested JEV in GrowthOS, Rippling’s agent harness for GTM, using anonymized real-world data.
E.g. a marketer or RevOps user could ask, “Classify these campaign emails by strategy,” then analyze which messaging patterns perform best.
JEV achieved 90% accuracy versus 50% for the LLM baseline and ran nearly 8x faster.
It was not a universal win. JEV performed poorly as a control mechanism inside our text-to-SQL agent loop because it lacked the semantic context needed to interpret internal business definitions.
JEV could see that a query returned rows, but not know whether those rows represented Rippling’s canonical definition of “qualified pipeline.” The LLM could inspect the relevant models, filters, and business definitions before deciding whether the analysis was complete.
The takeaway: JEV looks extremely promising for repeatable 'classification' like workloads, while LLMs remain better suited to context-heavy reasoning. But pairing them is pretty freakin awesome.
