VERIFYDevtools & code
Jev Experiment: LLM Anti-Pattern Linter
GrumpyBalkanSoftwareEngineer@TheBalkanHacker
1:15This project is a Jev experiment to create an inline code linter that identifies and mitigates anti-patterns commonly introduced by LLMs during code generation, providing near-instant feedback.
Ok I was busy doing actual work today, but here is another Jev experiment ... Code linting, for specifically the kind of "anti-patterns" that LLMs tend to include, "inline" as part of the code gen. process giving immediate feedback.
The idea here is that while you can create a set of ast/ or other programmatic rules for alot of these, the models tend to get really creative to satisfy the linter but still sneak in the "issue" in the code. You already know I'll say it, Goodhart's law!
But here the judgement can draw on the broader heuristics of large-ish model but obviously for cheaper and near instant compared to using another LLM. So this allow to use Jev in the tool call loop directly and mitigate the issues in the current set of changes causing issues in down stream code gen.
Kinda like the '80 movies where "only a ninja, can kill a ninja" ...
This is obviously just a quick test prototype but you can imagine doing something like this using @opencode plugins (or something like that) to hook into a real harness and its tool calls.
@typesafeai @dotpem