OpenAI is amending its hastily arranged deal to supply artificial intelligence to the US Department of War (DoW) after the ChatGPT owner’s chief executive admitted it looked “opportunistic and sloppy”.
The contract prompted fears the San Francisco startup’s AI could be used for domestic mass surveillance but its boss, Sam Altman, said on Monday night the startup would explicitly bar its technology from being used for that purpose or being deployed by defence department intelligence agencies such as the National Security Agency (NSA).
OpenAI, which has more than 900 million users of ChatGPT, made the deal almost immediately after the Pentagon’s existing AI contractor, Anthropic, was dropped.
Anthropic had insisted “using these systems for mass domestic surveillance is incompatible with democratic values”, leading the US president, Donald Trump, to call Anthropic “leftwing nut jobs” and directing the federal government to stop using its technology.
Despite denials from OpenAI that the agreement allowed for surveillance use, commentators raised the spectre of the Snowden scandal, which broke in 2013, when it emerged the NSA was engaged in mass harvesting of phone and internet communications.
The deal prompted an online backlash against OpenAI, with users of X and Reddit encouraging a “delete ChatGPT” campaign. One post read: “You’re now training a war machine. Let’s see proof of cancellation.”
Claude, the chatbot made by Anthropic, jumped to the top of Apple’s App Store charts, rising above ChatGPT, according to analysis by Sensor Tower.
In a message to employees reposted on X, the OpenAI CEO said the original deal announced on Friday had been struck too quickly after Anthropic was dropped.
“We shouldn’t have rushed to get this out on Friday,” Altman wrote. “The issues are super complex, and demand clear communication. We were genuinely trying to de-escalate things and avoid a much worse outcome, but I think it just looked opportunistic and sloppy.”
Upon announcing the deal, OpenAI had said the contract had “more guardrails than any previous agreement for classified AI deployments, including Anthropic’s”.
However, the use of AI by the US military has alarmed nearly 900 employees at OpenAI and Google, also a leading power in the technology, who have signed an open letter calling on their bosses to refuse to let the DoW use their products for surveillance and autonomous killing.
Warning that the US government was trying to “divide each company with fear that the other will give in”, they wrote: “We hope our leaders will put aside their differences and stand together to continue to refuse the DoW’s current demands for permission to use our models for domestic mass surveillance and autonomously killing people without human oversight.”
The letter has been signed by 796 Google employees and 98 OpenAI staff. OpenAI said in a blogpost announcing the DoW deal that one of its red lines was “no use of OpenAI technology to direct autonomous weapons systems”.
However, observers including OpenAI’s former head of policy research, Miles Brundage, have queried how OpenAI has managed to secure a deal that assuages ethical concerns Anthropic believed were insurmountable. Posting on X, he wrote: “OpenAI employees’ default assumption here should unfortunately be that OpenAI caved + framed it as not caving, and screwed Anthropic while framing it as helping them.”
Brundage added: “To be clear, OAI is a complex org, and I think many people involved in this worked hard for what they consider a fair outcome. Some others I do not trust at all, particularly as it relates to dealings with government and politics.”
In his X post, he also wrote that he would “rather go to jail” than follow an unconstitutional order from the government.
“We want to work through democratic processes,” Brundage wrote. “It should be the government making the key decisions about society. We want to have a voice, and a seat at the table where we can share our expertise, and to fight for principles of liberty.”
Meanwhile, three more US cabinet-level agencies – the departments of state, Treasury and health and human services – have moved to cease use of Anthropic’s AI products after the DoW’s declaration of the company as a supply chain risk. Trump has ordered all US government agencies to phase out their use of Anthropic after secretary of defence Pete Hegseth’s decision.
