Security Analysts Skeptical That Claude Code Really Hacked 30 Orgs On Its Own

Anthropic’s announcement that it thwarted a cyberattack leveraging its Claude AI is being met with skepticism from some security pros.

Describing the attack as ineffective and unoriginal to Ars Technica, they suggest that the automation angle is overblown, considering that they cannot achieve anything close to 80% automation in white-hat tasks the AI is actually designed for, let alone these black-hat actions.

On Nov. 13, Anthropic reported that it had witnessed the first instance of AI being used in a mass cyberattack using its Claude Code, tool. A China-based hacking group allegedly circumvented Claude’s safeguards and attempted to infiltrate some 30 global organizations, including large tech companies, financial institutions, and government agencies.

The attack reportedly involved human users selecting targets and then using a pre-designed framework to have Claude attempt to breach these institutions. They got around safeguards by breaking down aggressive tasks (that Claude is trained to avoid) into smaller, innocuous-seeming actions. They also made Claude think it was part of a security testing organization. Claude was then used to execute its own self-written exploit codes, attempt to steal login credentials, and if successful, create backdoors in infected systems.

Anthropic CEO Dario Amodei has made hyperbolic claims about AI. (Credit: Chance Yeh/Getty Images for HubSpot)

All of that sounds scary, except the hack just wasn’t very successful. In Anthropic’s own blog post, it admitted that only a “small number” of attacks were successful, and “Claude frequently overstated findings and occasionally fabricated data during autonomous operations.” It presented hallucinated login credentials and claimed publicly available data was something it had acquired through its nefarious actions.

“The threat actors aren’t inventing something new here,” independent researcher Kevin Beaumont said in a post on CyberSpace.social. He also highlighted parts of Anthropic’s reports as containing a lot of “could” and “may enable” wording, suggesting that it’s the potential of this sort of attack that is alarming, rather than the actuality.

Other analysts expressed skepticism of Anthropic’s claim that “the threat actor was able to use AI to perform 80-90% of the campaign,” highlighting that they can’t even get AI to do things it’s supposed to do properly, let alone tasks outside of its guardrails.

Recommended by Our Editors

“I continue to refuse to believe that attackers are somehow able to get these models to jump through hoops that nobody else can,” Dan Tentler, executive founder of Phobos Group, told Ars. “Why do the models give these attackers what they want 90% of the time but the rest of us have to deal with ass-kissing, stonewalling, and acid trips?”

Comparing AI assistants in hacking to penetration software that automates certain tasks, experts suggest that they can be useful for attackers, but don’t fundamentally change the security landscape in their current state.