✦ Emerging

Agent trace evaluation with typed judges

Typed decision models like Jev are used to evaluate agent traces, offering a cost-effective and fast alternative to LLM judges.

9 builds · 9 builders · First seen Sep 20 · 5.3× momentum

VulcanBench Verdict Eval Suite

VulcanBench Verdict is an evaluation suite designed for System One Models, utilizing Jev for rapid testing.

Morgan@morganlinton

1.6K views on XJev score 45

Jev Agent Evaluation

Jev is an agent evaluation tool that uses LLM judges to assess agent performance, offering a low-cost and accurate alternative to other evaluation methods.

Sai Teja@saiitoshii

33 views on XJev score 69

Jev for LangSmith Evals

Jev is a System-One judge that scores agent traces in real-time within LangSmith Evals, supporting typed yes/no, choice, and score outputs with probabilities.

AI Alerts@aialertsai

16 views on XJev score 47

Jev System One Model Evaluation

This project evaluates Jev, a System One model, against standard LLM judges on accuracy, repeatability, latency, and cost, exploring the trade-off between fast, pattern-matching reasoning and slow, step-by-step generation for agent evaluation.

Read the Diff@read__the__diff

73 views on XJev score 58

Jev as an Agent Judge Benchmark

LangChain evaluated Jev, a model that provides typed answers and probabilities, as an agent judge, finding it superior in accuracy and cost compared to LLM judges like Claude and GPT-5.6.

AI Horizon Blog@AIHorizonBlog

Posted 22h agoJev score 72

Jev Evals

Jev evals replace expensive LLM judges with typed Jev decisions, enabling faster and cheaper evaluations within agent loops.

AiIndigo@IndigoYogiArt

4 views on XJev score 64

Jev LLM Judge Performance

Jev is a competitive LLM judge that identifies agent scope violations efficiently and cost-effectively.

dreadnode@dreadnode

2.8K views on XJev score 46

Typed Evals: LLM Evaluation Framework

Typed Evals is an open-source Python framework designed for evaluating LLMs, RAG pipelines, and AI agents by leveraging typed judge backends such as Jev.

Aaryan Verma@iam_aaryanverma

45 views on XJev score 48

Jev Agent Evaluation

This project evaluates Jev against LLM judges on accuracy, repeatability, latency, and cost to determine if System One models can provide a novel method for agent evaluation.

LangChain@LangChain

19K views on XJev score 44