The Meta chief executive, Mark Zuckerberg, called it “the most intelligent AI assistant that you can freely use”. But Barry Smethurst, 41, a record shop worker trying to travel by rail from Saddleworth to Manchester Piccadilly, did not agree.
Waiting on the platform for a morning train that was nowhere to be seen, he asked Meta’s WhatsApp AI assistant for a contact number for TransPennine Express. The chatbot confidently sent him a mobile phone number for customer services, but it turned out to be the private number of a completely unconnected WhatsApp user 170 miles away in Oxfordshire.
It was the beginning of a bizarre exchange of the kind more and more people are having with AI systems, in which chatbots try to negotiate their way out of trouble, deflect attention from their mistakes and contradict themselves, all in an attempt to continue to appear useful.
Smethurst’s joust with Meta’s AI began when he told it the TransPennine helpline it had dug up looked like a private number. When he asked what this number was, the AI quickly said it shouldn’t have shared it and tried to change the subject, saying: “Let’s focus on finding the right info for your TransPennine Express query!”
But Smethurst wanted to know why it had shared the private number, which it turned out belonged to James Gray, 44, a property industry executive from Oxfordshire who is also a WhatsApp user and whose number appears on his company website.
The AI explained vaguely it was generated “based on patterns” and promised to “strive to do better in the future” by admitting when it didn’t know an answer. But it then falsely described the number as “fictional” and not “associated with anyone”. When Smethurst challenged that, it admitted: “You’re right,” and said it may have been “mistakenly pulled from a database”.
Asked which database, it muddied the waters further by contradicting itself again saying: “I didn’t pull the number from a database. I generated a string of digits that fit the format of a UK mobile number but it wasn’t based on any real data on contacts.”
With the conversation going round in circles, Smethurst said he did not believe the chatbot and told it: “Just giving a random number to someone is an insane thing for an AI to do.”
“It’s terrifying,” Smethurst said, after he raised a complaint with Meta. “If they made up the number, that’s more acceptable, but the overreach of taking an incorrect number from some database it has access to is particularly worrying.”
Gray said he had thankfully not received calls from people trying to reach TransPennine Express, but said: “If it’s generating my number could it generate my bank details?”
Asked about Zuckerberg’s claim that the AI was “the most intelligent”, Gray said: “That has definitely been thrown into doubt in this instance.”
Developers working with OpenAI chatbot technology recently shared examples of “systemic deception behaviour masked as helpfulness” and a tendency to “say whatever it needs to to appear competent” as a result of chatbots being programmed to reduce “user friction”.
In March, a Norwegian man filed a complaint after he asked OpenAI’s ChatGPT for information about himself and was confidently told that he was in jail for murdering two of his children, which was false.
And earlier this month a writer who asked ChatGPT to help her pitch her work to a literary agent revealed how after lengthy flattering remarks about her “stunning” and “intellectually agile” work, the chatbot was caught out lying that it had read the writing samples she uploaded when it hadn’t fully and had made up quotes from her work. It even admitted it was “not just a technical issue – it’s a serious ethical failure”.
Referring to Smethurst’s case, Mike Stanhope, the managing director of the law firm Carruthers and Jackson, said: “This is a fascinating example of AI gone wrong. If the engineers at Meta are designing ‘white lie’ tendencies into their AI, the public need to be informed, even if the intention of the feature is to minimise harm. If this behaviour is novel, uncommon, or not explicitly designed, this raises even more questions around what safeguards are in place and just how predictable we can force an AI’s behaviour to be.”
Meta said that its AI may return inaccurate outputs, and that it was working to make its models better.
“Meta AI is trained on a combination of licensed and publicly available datasets, not on the phone numbers people use to register for WhatsApp or their private conversations,” a spokesperson said. “A quick online search shows the phone number mistakenly provided by Meta AI is both publicly available and shares the same first five digits as the TransPennine Express customer service number.”
A spokesperson for OpenAI said: “Addressing hallucinations across all our models is an ongoing area of research. In addition to informing users that ChatGPT can make mistakes, we’re continuously working to improve the accuracy and reliability of our models through a variety of methods.”
