Study Unequivocally Points to Inferiority of AI Knowledge Search
Participants were then asked to write advice for a friend on the subject that they’d just learned about. The results showed that the group who had used an LLM felt that they had learned less, and subsequently invested less effort in writing their advice, which was consistently shorter, less factual, and more generic.
Both sets of advice were presented to an independent sample of readers, who were unaware of which had come from internet search versus AI use. They found the advice that came from AI search was less helpful, less informative, and they were less likely to adopt it.
In an effort to control as many variables as possible, another experiment was run in which participants were exposed to the same set of facts, regardless of whether they came from Google search or ChatGPT. The researchers theorized that AI users were potentially exposed to a less eclectic range of information, owing to the way that LLMs scrape information. The results still indicated that AI search led to a shallower retention and recreation of knowledge.
