RANKOther
AI Pipeline Evaluation for Rheumatology Cases
RheumAI@RheumaAI
This project tested four AI pipelines on synthetic rheumatology cases, with Deep ORVS + Jev ranking highest and outperforming an OpenAI baseline, though with higher latency. The mechanism of Jev is unspecified.
We tested 4 AI pipelines on 20 synthetic rheumatology cases. The automated judge ranked Deep ORVS + Jev #1 in 12/20 and above the OpenAI baseline in 17/20. Trade-off: 14.9s vs 6.8s latency. Early signal, not clinical validation. @typesafeai @troopdegen