Whose experience is missing from the demonstration?
A smooth demonstration cannot show every person’s experience. Research on inconsistent AI responses prompts a closer look at the tasks, language, access needs, and repeated attempts included in a trial.
- Availability
- Published
- For
- Readers
What happens next
An October 2023 study in npj Digital Medicine tested how four language-model systems answered selected questions about race and medicine. The study concerned particular medical questions and model versions. It cannot establish how a community tool performs. It does show why one convincing response is a limited basis for confidence, especially when a system may behave differently across attempts.