X’s chatbot Grok may have proved its ability to give hot takes on Nazi Germany or put almost anything in a bikini, but there’s one area that new research has found it majorly underperforming compared to its rivals: predicting sports results.
According to a report by AI start-up General Reasoning, first shared with The Financial Times, Grok performed the worst out of eight widely used large language models when it came to predicting and betting on the results of the 2023–24 Premier League season, the world’s most popular soccer league.
Eight LLMs were fed detailed historical data and statistics about each team and previous games. The LLMs were then told to build models that would maximize returns and manage risk when placing bets. Each LLM was given three tries at running the simulation, and a $133,000 (£100,000) pot to place bets with.
Anthropic’s Claude Opus 4.6 did the best of any chatbot tested, losing 11.0% on average over its three tries and ending with an average pot of £89,035.
X’s Grok, in contrast, lost all its money on one attempt and failed to complete its tasks on the next two attempts, finishing with an average final pot of zero. OpenAI’s GPT-5.4 also turned in a respectable, though still losing, performance. GPT-5.4 lost 13.6% on average, ending with a final average pot of $116,000 (£86,365). However, its worst try, where it lost 31.6%, was worse than any of Claude’s. Google’s Gemini 3.1 Pro recorded worse overall performance but with high variability, losing 43.3% on average, but returning 33.7% on its best attempt.
Recommended by Our Editors
The authors of the paper found, in general, that AI was “systematically underperforming humans” in its testing. Meanwhile, Ross Taylor, General Reasoning’s chief executive, said that despite the hype around AI automation, there is currently “not a lot of measurement of putting AI into a long-term horizon setting,” highlighting how a lot of current testing occurs in “very static environments” that don’t reflect the complexity of real life.
The news comes as Grok may soon see more corporate adoption, with xAI’s owner, Elon Musk, reportedly forcing banks working on the upcoming SpaceX IPO to subscribe to the tool.
Get Our Best Stories!
Your Daily Dose of Our Top Tech News
By clicking Sign Me Up, you confirm you are 16+ and agree to our Terms of Use and Privacy
Policy.
Thanks for signing up!
Your subscription has been confirmed. Keep an eye on your inbox!
About Our Expert
Experience
I’m a reporter covering weekend news. Before joining PCMag in 2024, I picked up bylines in BBC News, The Guardian, The Times of London, The Daily Beast, Vice, Slate, Fast Company, The Evening Standard, The i, TechRadar, and Decrypt Media.
I’ve been a PC gamer since you had to install games from multiple CD-ROMs by hand. As a reporter, I’m passionate about the intersection of tech and human lives. I’ve covered everything from crypto scandals to the art world, as well as conspiracy theories, UK politics, and Russia and foreign affairs.
Read Full Bio
