CHOOSEBusiness workflows
Jev Classification Experimentation
More Donut than Man 🍩@jamiegrove
This project explores the capabilities of Jev, a classification tool, by testing its performance on real-world e-commerce data across various dimensions and session types, aiming to understand its utility for classifying completed sessions.
TakeLike most classifiers, it depends on whether the evidence you feed it contains something useful to classify.
I did Bayesian classifier work in the stone ages some 30 years ago. Looking at Jev now I’m actually pretty excited. Like most classifiers, it depends on whether the evidence you feed it contains something useful to classify. A probability attached to an answer doesn’t create information that wasn’t there.
I’ve spent the last few days testing Jev against real ecommerce behavior. Simple rules beat it on sparse purchase-intent signals. A more promising result turned up when I explored categorical outputs, then disappeared when I froze the test and ran it on a separate sample.
That’s useful to know before you build infrastructure around a flattering number.
The part that interests me is classifying completed sessions across multiple dimensions in a single call. Five hundred sessions, eight dimensions, about three minutes. The labels looked sensible on inspection, although that isn’t the same as establishing accuracy.
You can hand Jev a bounded piece of work, ask several related questions, and get structured results back without filling the calling agent’s context with the whole exercise.
Thirty years ago, I would have spent a lot of time building the machinery around that. Making it this easy to try changes which problems are worth investigating.
You still have to define the categories, supply useful evidence, check whether the outputs discriminate, and compare them with a baseline. The judgment API makes the experiment cheap. It doesn’t relieve you of the experiment.
That’s enough to get me excited.