CHOOSEAgents & automation
JevHarness for AI Automation
21Borov@21_borv
JevHarness is an AI automation framework that separates roles between deep thinking during development and quick, non-strict decision-making at runtime using a component called Jev.
TakeThe scheme looks like this: task → LLM designs harness → code prepares state → Jev selects → code validates and performs action.
JevHarness skill
JevHarness offers an unusual scheme for AI automation.
A strong LLM does not make every decision in runtime. She designs a specific workflow once.:
- what signs can be extracted from the input data;
- What options are acceptable?;
- what is considered a good result;
- where deterministic logic is needed;
- where a fast probabilistic choice is sufficient.
After that, the workflow is fixed.
The usual code considers the facts and controls the permitted actions. Jev makes quick structured decisions: choice, score or noul. And the heavy author model is no longer invoked at every step.
The scheme looks like this:
task → LLM designs harness → code prepares state → Jev selects → code validates and performs action.
The authors call this a separation of roles:
1) think deeply when developing,
2)fix the strategy,
3)Use Jev for quick, non-strict solutions.
There is a Pokémon example in the repository. After five rounds of reflection, the authors improved the win rate on their Eval set from 3 out of 12 to 9 out of 12.
But the repository shows well what needs to be optimized.
Harness can store:
- feature engineering;
- instructions and criteria for Jev;
- graph of dependent solutions;
- process memory;
- acceptable actions;
- full traces;
- reward and verification results.
If there is a reliable evaluator, JevHarness is able to transfer the full execution paths and rewards to LLM so that it offers an improved version of the pipeline. GEPA is used for the search, and the selected option can be frozen along with the task's runtime, specification, and resources.
GitHub:
github.com/TianyuCodings/JevHarness
Related builds
