Will computers ever match or surpass human-level intelligence – and if so, how? When the Association for the Advancement of Artificial Intelligence (AAAI), based in Washington DC, asked its members earlier this year whether neural networks – the current star of artificial intelligence systems – alone will be enough to achieve this goal, the vast majority said no. Instead, most say, getting these systems up to par will require a heavy dose of an older breed of AI: symbolic AI.
Will AI ever win its own Nobel Prize? Some predict an award-worthy scientific discovery soon
Symbolic AI, also known as ‘good old-fashioned AI’, is based on formal rules and an encoding of the logical relationships between concepts1. For example, mathematics is symbolic, as are “if-then” statements and computer code languages like Python, along with flowcharts or Venn diagrams that map out how, for example, cats, mammals, and animals are conceptually connected. Decades ago, symbolic systems were an early frontrunner in AI efforts. However, in the early 2010s they were far surpassed by more flexible neural networks. These machine learning models excel at learning from large amounts of data and underlie large language models (LLMs) and chatbots such as ChatGPT.
Now, however, the computer science community is pushing hard for a better and bolder merger of old and new. “Neurosymbolic AI” has become the hottest buzzword in town. Brandon Colelough, a computer scientist at the University of Maryland at College Park, has charted the concept’s meteoric rise in academic papers (see “Up and Up”). These reveal a spike in interest in neurosymbolic AI that started around 2021 and shows no signs of slowing down2.
Many researchers are heralding the trend as a breakaway from what they see as an unhealthy monopoly of neural networks in AI research, and expect the shift to produce smarter and more reliable AI.
A better merger of these two strategies could lead to artificial general intelligence (AGI): AI that can reason and generalize its knowledge from one situation to another, just like humans do. It could also be useful for high-risk applications such as military or medical decision-making, Colelough says. According to him, because symbolic AI is transparent and understandable to humans, it does not suffer from the ‘black box’ syndrome, which makes neural networks difficult to trust.
Source: updated from ref. 2
There are already good examples of neurosymbolic AI, including Google DeepMind’s AlphaGeometry, a system reported last year3 that can reliably solve Math Olympiad problems – questions aimed at talented high school students. But figuring out how best to combine neural networks and symbolic AI into an all-purpose system is a huge challenge.
“You’re really designing these kinds of two-headed beasts,” says computer scientist William Regli, also at the University of Maryland.
War of words
In 2019, computer scientist Richard Sutton posted a short essay entitled “The bitter Lesson” on his blog (see go.nature.com/4paxykf). In it, he argued that since the 1950s, people have repeatedly assumed that the best way to make intelligent computers is to feed them all the insights humans have come to about the rules of the world, in areas from physics to social behavior. The bitter pill we have to swallow, Sutton wrote, is that symbolic methods are being surpassed time and time again by systems that use a lot of raw data and scaled-up computing power to leverage “search and learn.” For example, early chess computers, which were trained in human-devised strategies, outperformed computers that were simply fed lots of game data.
This lesson has been widely cited by proponents of neural networks to support the idea that making these systems bigger and bigger is the best path to AGI. But many researchers argue that the essay overstates the case and downplays the crucial role that symbolic systems can play in AI. For example, today’s best chess program Stockfishlinks a neural network to a symbolic tree of allowed moves.
AI Models That Lie, Cheat, and Plot Murder: How Dangerous Are LLMs Really?
Neural nets and symbolic algorithms both have advantages and disadvantages. Neural networks consist of layers of nodes with weighted connections that are adjusted during training to recognize patterns and learn from data. They are fast and creative, but they also have to make things up and cannot provide reliable answers to questions beyond the scope of their training data.
Symbolic systems, meanwhile, struggle to encompass “messy” concepts such as human language, involving huge rule databases that are difficult to build and slow to search. But their methods are clear, and they are good at reasoning, using logic to apply their general knowledge to new situations.
When used in the real world, neural networks without symbolic knowledge make classic mistakes: image generators can draw people with six fingers on each hand because they have not learned the general concept that hands typically have five fingers; video generators have trouble bouncing a ball around a scene because they haven’t learned that gravity pulls things down. Some researchers blame such errors on a lack of data or computing power, but others say the errors illustrate the fundamental inability of neural networks to generalize knowledge and reason logically.
Many argue that adding symbolism to neural networks may be the best – and even the only – way to inject logical reasoning into AI. For example, global technology company IBM endorses neurosymbolic techniques as a path to AGI. But others remain skeptical: Yann LeCun, one of the founders of modern AI and chief AI scientist at tech giant Meta, has said that neurosymbolic approaches are “incompatible” with neural network learning.
Can AI really be creative?
Sutton, from the University of Alberta in Edmonton, Canada, who won the 2024 Turing Prize, the equivalent of the Nobel Prize for computer science, stands by his original argument: “The bitter lesson still applies to today’s AI,” he said. Nature. This suggests, he says, that “adding a symbolic, more manually crafted element is probably a mistake.”
Gary Marcus, an AI entrepreneur, writer, and cognitive scientist based in Vancouver, Canada, and one of the most outspoken proponents of neurosymbolic AI, tends to view this disagreement as a philosophical battle now being settled in his favor.
Others, like roboticist Leslie Kaelbling of the Massachusetts Institute of Technology (MIT) in Cambridge, say arguments about which view is correct are a distraction, and people should just keep doing what works. “I’m a magpie. I’ll do anything that makes my robots better.”
Mix and match
Apart from the fact that neurosymbolic AI aims to combine the benefits of neural nets with the benefits of symbolism, its definition is vague. Neurosymbolic AI encompasses “a very big universe,” says Marcus, “of which we have only explored a small part.”
There are many broad approaches, which people have tried to categorize in different ways. One option highlighted by many is the use of symbolic techniques to improve neural nets4. AlphaGeometry is perhaps one of the most advanced examples of this strategy: it trains a neural net based on a synthetic dataset of mathematical problems produced using a symbolic computer language, making the solutions easier to check and fewer errors. It combines the two elegantly, says Colelough. In another example, ‘logical tensor networks’ provide a way to code symbolic logic for neural networks. A fuzzy truth value can be assigned to statements5: a number somewhere between 1 (true) and 0 (false). This provides a framework of rules to help the system reason.
Another broad approach does what some would say the opposite, using neural nets to refine symbolic algorithms. A problem with symbolic knowledge bases is that they are often so large that searching takes a very long time: for example, the ‘tree’ of all possible moves in a game of Go contains about 10 moves.170 positions that are unfeasibly large to get through. Neural networks can be trained to predict the most promising subset of moves, allowing the system to shorten how much of the ‘tree’ it has to search, thus speeding up the time it takes to arrive at the best move. That’s what Google’s AlphaGo did when it famously outperformed a Go grandmaster.
