If you could place a bet on a chess match-up between the Atari 2600 and ChatGPT, which technology would you put your money on? The Atari 2600 is a game console released in 1977, 48 years ago. It features an 8-bit processor running at a clock speed of 1.19MHz. That’s megahertz, folks, not gigahertz. The Atari machine carries 128 bytes of RAM. According to infrastructure architect Robert Caruso, who posted the results back in June, had you bet on ChatGPT, you would have lost.
Who would have thought a 48-year-old Atari 2600 could crush ChatGPT and Copilot in chess?
Caruso wrote that “ChatGPT got absolutely wrecked on the beginner level.” He also explained how this apparent upset could take place. “Despite being given a baseline board layout to identify pieces, ChatGPT confused rooks for bishops, missed pawn forks, and repeatedly lost track of where pieces were,” he stated. ChatGPT recognized that its loss was an upset and blamed it on the abstract icons used by Atari to represent chess pieces. However, even after changing to standard chess notation, the chatbot “made enough blunders to get laughed out of a 3rd grade chess club.”
Caruso said that during an hour and a half, he had to stop ChatGPT from making bad moves and help it understand where the pieces belonging to both sides were located. Several times per turn ChatGPT had to have its awareness of the board reset. The AI chatbot promised multiple times that its playing would improve if the game started over from the beginning. Finally, ChatGPT conceded the match.
Interestingly, it was ChatGPT that suggested the contest. During a conversation about chess, ChatGPT said that it was a strong player and could easily beat Atari Video Chess, which thinks only 1-2 moves ahead. ChatGPT was curious about how fast it could win. Instead, we know it lost.
At the start of this month, the Atari 2600 game console took on another modern AI powerhouse in a game of chess. Microsoft’s Copilot thought that the Atari 2600 was no match for it. Even though it said that it could think 10-15 moves ahead of the Atari, Copilot said that it would stick to thinking 3-5 moves ahead because the Atari console makes “suboptimal moves. Copilot also said that, unlike ChatGPT, “I make a strong effort to remember previous moves and maintain continuity in gameplay, so our match should be much smoother.”
As it turned out, Caruso had to send Copilot a screenshot of the board after every move the Atari 2600 made. The Microsoft AI chatbot admitted that it had the same spatial memory gaps as ChatGPT. Despite this shortcoming, Copilot said that it could still analyze the board and make good moves. Well, apparently it couldn’t as the Atari 2600 made it two straight victories against an AI opponent.
Gemini decided that the best move was to cancel its match with the Atari 2600
While Gemini never did get a chance to take on the Atari 2600, Caruso was impressed with how the chatbot had the insight to know its own limitations.
It seems incredible that a game console 48 years old using woefully underpowered technology, could beat two AI chatbots with cutting-edge tech and scare a third one into cancelling a match. But this is not really the upset that you might think it is. Senior Software Engineer Kyle Witeck wrote, “So…you’re comparing language models to a chess bot that was designed to play chess…classic…AI is not intelligent and not AI. It patterns and guesses from a system designed on language. Token-driven guesses.”
