People often trust AI answers too quickly

Researchers from the Canadian University of Waterloo and University College London have investigated possible differences in communication between humans and between humans and machines. The aim was to find out how answers from AI systems are evaluated compared to those of other people in terms of security and self-confidence.

People overestimate AI responses

It turned out that the test subjects systematically overestimated the confidence of AI systems compared to that of humans – even when their behavior was identical. The results of the study were published in Communications Psychology.

“As AI systems become more sophisticated, people increasingly rely on them for advice, such as what products to buy or what content to consume,” Clara Colombatto of the Department of Psychology at the University of Waterloo told phys.org.

A similar trust is reflected in human interaction, where one seeks guidance and advice from others, says Colombatto. An important difference, however, is that when giving advice, people often communicate how confident they are in their assessment. According to the psychologist and study author, it depends on how much you trust and rely on the advice.

AI systems lack self-assessment

It is precisely this self-assessment that is missing from AI systems such as ChatGPT or Gemini. These generated answers to certain questions without indicating how sure they were about them. As part of the research, Colombatto and her colleagues found that people derived their trust in AI competence from how quickly the respective answer came or how capable the system was generally perceived to be.

The consequence of this self-conclusion: “People can end up placing too much trust in AI recommendations, even if the system is insecure or error-prone,” warns Colombatto. The researchers are now working on giving people more support so that they can better recognize the skills of AI systems in the future.