CHOOSEBusiness workflows
Tried @typesafeai's semantic-find cookbook on supplier replies for Factory Twelve, an apparel so
Jeff Fedor@a_jfedor
TakeCode splits the source into clauses, preserves source IDs, versions, hashes and character offsets, then copies the selected text verbatim.
Tried @typesafeai's semantic-find cookbook on supplier replies for Factory Twelve, an apparel sourcing platform. Interesting first results with jev-1.13.0.
I composed three questions over the same state in one System One request:
- Choice — `where`: select the most relevant clause ID.
- Noul — `exists`: probability that the reply supplies at least one requested detail.
- Choice — `coverage`: classify the answer as `complete`, `partial`, or `absent`.
Keeping existence separate matters because the clause Choice always produces a winner. Keeping coverage separate matters because a Noul near 0.5 represents uncertainty, not a partially complete answer.
Code splits the source into clauses, preserves source IDs, versions, hashes and character offsets, then copies the selected text verbatim. No generated quotations.
First run: 12 synthetic development cases
- Existence: 12/12 correct at a 0.5 threshold.
- Clause selection: 8/8 correct on answer-bearing cases.
- No highlights shown for all 4 absent-answer cases.
- Coverage: 9/12 correct.
- Mean request latency: 455 ms.
- Total usage: 9,742 input / 1,548 output tokens.
The three coverage errors all had correct clause selection:
- `$8.50 per piece`: `complete`, despite an unspecified named currency.
- `USD 8.50`: `complete`, despite an unspecified price basis.
- `30% deposit, 70% before shipment`: `partial`, where our rubric accepts the schedule as complete.
It correctly treated “third-party testing costs extra” as a complete answer to whether testing is included. Answer completeness and requirement compliance are different judgments.
For display, code requires non-absent coverage plus Noul ≥0.7 to highlight evidence; absent coverage plus Noul ≤0.3 shows no answer; disagreement goes to review. Those thresholds are provisional and uncalibrated but based on other experiments.
Clause selection looks promising as a small, composable primitive which is super exciting.
Unsurprisingly, completeness needs a tighter contract, potentially decomposed into individual requested details and combined with deterministic checks.
This is based off a small synthetic set, no tuning after this run, and no holdout evaluation yet.
How others are decomposing coverage: one Choice over the whole query, or independent Nouls for each required detail?