CHOOSEOther
Jev Probability Assessment Accuracy
Simon Smith@_simonsmith
This project evaluates the accuracy of Jev's probability assessments across different response types, finding that 'Choice' with numerical answers yields the most correct probabilities.
TakeTL; DR: Use "Choice" with numerical answers if you want Jev to select the correct probabilities.
Something important about Jev: How you ask probability questions affects accuracy.
I saw people on X posting about Jev's incorrect probability assessments for things like coin tosses and dice rolls, so I had Astra prepare a 60-question test on coins, dice, cards and other problems with known mathematical answers, and run it with Jev's different response types.
TL; DR: Use "Choice" with numerical answers if you want Jev to select the correct probabilities.
Otherwise, Noul is second best, but will return incorrect probabilities, like a 55% probability of a fair coin toss landing heads.
