VERIFYAgents & automation
LLM Intent Checker with Jev Grading
Nathan Oyler@NathanOyler
This project adds an intent checker to LLM requests, using Jev to grade the LLM's interpretation against the original request to ensure it aligns with user intent.
I promise I am not trying to use Jev everywhere, but I stumbled on an idea that is amazing right now.
The best models don't understand what you're asking. It gets worse the higher the effort level is. This is because they are trained to do more and more as autonomous agents.
It drives me insane, because I want exactly what I asked for.
I added an intent checker on my requests. It takes my request, what the LLM states is what I wanted, and then jev grades it with 50 questions.
What happens is, every, and I mean every, single request, the LLM thinks I want all sorts of stuff I didn't ask for, and Jev keeps it in check.
I am so happy right now. It is so frustrating asking for an apple, and getting back an orchard.
Related builds
