Artificial intelligence is getting better at mimicking human language, solving problems, and even passing exams. But according to new research, it still can’t replicate one of the most fundamental parts of human cognition—how humans think.
A recent study published in Transactions on Machine Learning Research examined how well large language models, like OpenAI’s GPT-4, handle analogical reasoning. The results found that while humans had no trouble applying general rules to letter-based problems—such as spotting a repeated character and removing it—AI systems consistently missed the mark.
The researchers say the issue wasn’t that the AI lacked data. Instead, it was that it couldn’t generalize patterns beyond what it had already been taught. This exposes a key difference in how humans and AI think.
Humans are remarkably good at abstract reasoning. We can take a concept we’ve learned in one context and apply it in a completely new one. We understand nuance, adapt to unfamiliar rules, and build mental models of how things should work. AI, on the other hand, relies heavily on memorizing patterns from massive amounts of data. That helps it predict what comes next—but not why it comes next.
The implications here are massive for the future of AI. In fields like law, medicine, and education—where analogy and contextual understanding are crucial—AI’s limitations could lead to errors with real consequences. The difference in how humans and AI think is just too great.
For example, a human might recognize that a new legal case closely mirrors an older precedent, even if the wording is different. However, an AI might miss that entirely if the phrasing doesn’t align with its training data. This could lead to huge issues with legal ramifications.
And this isn’t just a technical quirk. It’s ultimately a foundational divide. Yes, AI can simulate human responses. However, that’s not the same as thinking like a human. This is one reason AI will never be as good at creative writing as humans are, despite what OpenAI’s CEO might say. Plus, the more we rely on these systems, the more important it becomes to understand what they can’t do, especially if studies are right and we’re losing our critical thinking skills because of AI usage.
OpenAI’s new o1-pro reasoning model might be the best on the market, but if it can’t think like a human, then it will never be able to replace humans. As the study’s authors put it, accuracy alone isn’t enough. We need to be asking tougher questions about how robust AI really is when the rules aren’t written down—and whether we’re ready for the consequences if it gets them wrong.
