Can you tell if you’re chatting with a human or a chatbot? According to a new study, most people can’t. In fact, one of today’s top artificial intelligence models, OpenAI’s GPT-4.5, was judged to be human more often than actual humans when it took part in an updated version of the classic AI Turing test.
For years, the Turing test has been the go-to for testing the intelligence of a machine, as well as just how humans perceive them. So it really isn’t surprising that researchers from UC San Diego’s Language and Cognition Lab ran a multi-participant version of the Turing test involving nearly 300 people.
In each round, participants engaged in side-by-side conversations—one with a human and one with an AI—and then had to decide which was human and which was AI. The results of this updated Turing test were pretty telling, too.
When GPT-4.5 was instructed to adopt a persona, like a pop-culture-savvy young adult, it fooled participants 73 percent of the time. That’s far above the 50 percent chance mark, which historically defines a “pass” in the AI Turing test. By comparison, actual humans were less successful at convincing participants of their humanity.
The experiment also tested other models, including Meta’s LLaMA 3 and OpenAI’s newer GPT-4o, along with ELIZA, one of the earliest chatbots ever developed. Unsurprisingly, models without a persona prompt performed much worse. GPT-4.5’s accuracy dropped to 36 percent without a defined character to inhabit, and GPT-4o scored a mere 21 percent.
This isn’t the first time researchers have put AI through the Turing test. But these results show that while language models might not “think” like humans do, they’re increasingly capable of sounding human in short, casual conversations. That has major implications—not just for proving AI intelligence, but also for understanding how easily people can be convinced that AI is actually a human.
Of course, the researchers caution that passing an AI Turing test doesn’t mean a model understands anything in the way humans do. However, it does mean these systems are becoming more and more skilled at performing human-like interactions, especially when given specific roles or tones to adopt.
This means it could become harder to spot AI in everyday situations. That, in turn, is raising concerns about misinformation, impersonation, and the overall future of human-AI interaction. The problem then becomes that we might not realize we’re interacting with an AI, which could have some serious ethical implications.
Plus, with each new generation of AI, the line between machine mimicry and human conversation is only going to get blurrier. AI is as bad as it is going to get right now. From here, the systems are only going to continue to improve. For now, these results are published on a preprint server. However, the researchers have submitted their paper to an official peer-reviewed paper, so we could see an official release sometime in the future.
