CHOOSEDevtools & code
JEV Model State Sensitivity
Sergii Makarevych@sermakarevich
This project investigates the sensitivity of a new JEV model to the emotional state of the user, observing how varying tones affect its responses to a question about Python type annotations.
Interesting observation about new JEV model by @typesafeai - it might be sensitive to the state more than I would like it too.
The test is, I ask a generic question and change the tone of the state to see how it affects the output.
Question: "Would you recommend using type annotations in Python code?"
State:
- positive
- mild_positive
- neutral
- mild_negative
- negative
Choice: Yes/ No / Unknown
Result: JEV returns all type of answers.
I think I would expect it to have a stable opinion on a matter which is less affected by a state. If the conclusion from the training set is that type annotation is good, it should not be that easy to shake this conclusion, I think. What do you think?