We aren't running Jev through our formal benchmarks because many of our evaluations involve open

Sep 19

SignalI gave Jev some proprietary data alongside OHLCV over the last 2 years and a few actions it could choose from (long/flat/short).

We aren't running Jev through our formal benchmarks because many of our evaluations involve open-ended design. But I was curious about its capabilities, so I ran it through some tests with my old quant trading infrastructure. I gave Jev some proprietary data alongside OHLCV over the last 2 years and a few actions it could choose from (long/flat/short). It's likely that some recent price data leaked into Jev's pre-training corpus (even though @typesafeai says their data is mostly synthetic), but I still thought it did a decent enough job to be worth running a more controlled experiment. It makes reasonable, patient decisions and avoids overtrading (which other LLMs we've tested struggle with). It's consistent, cheap, and fast. I definitely see the practical use cases for this model. Next step: obfuscating the symbol names, prices, and dates, and running it again.

Original post