LlamaFirewall is a security framework aimed at safeguarding AI agents against prompt injection, goal misalignment, and insecure code generation. It achieved over 90% efficacy in reducing attack success rates when evaluated on the AgentDojo benchmark. Additionally, developers can update its behavior by adding new security guardrails.
LlamaFirewall is a real-time guardrail monitor designed to serve as a final layer of defense against security risks associated with AI Agents. It includes three protection layers: PromptGuard 2, a universal jailbreak protector; Agent Alignment Checks, a chain-of-thought auditor that inspects agent reasoning for prompt injection and goal misalignment; and CodeShield, an online static analysis engine aimed at preventing the generation of insecure or dangerous code by coding agents.
PromptGuard 2 is a fine-tuned BERT-style model designed to detect jailbreak attempts that analyzes both user prompts and untrusted data sources in real time. It specifically addresses jailbreak tactics such as instruction overrides and token injection.
These techniques are often explicit, repetitive, and pattern-rich, making them more amenable to pattern-based detection approaches. Compared to goal hijacking attacks, jailbreaks exhibit higher lexical regularity and structural predictability. This characteristic also makes them a common entry point for novice attackers or automated adversarial tools.
In comparison with its previous generation, PromptGuard 2 brings performance improvements to the 86M parameter variation and lower latency to the lightweight 22M parameter variant.
AlignmentCheck is an experimental chain-of-thought auditor that inspects an agent’s reasoning to identify signs of goal hijacking or misalignment.
Instead of inspecting individual messages, it reasons over the entire execution trace, flagging deviations that suggest covert prompt injection, misleading tool output, or other forms of goal hijacking.
According to Meta’s researchers, it is the first open source guardrail capable of auditing the chain-of-thought of a large language model in real time, specifically designed for injection defense.
CodeShield is an online static analysis engine for LLM-generated code that supports both Semgrep and regex-based rules. Designed for extensibility, it enables syntax-aware pattern matching across eight programming languages to detect potential risks. Originally released as part of the Llama 3 launch, CodeShield is now integrated into LlamaFirewall.
Although CodeShield is effective in identifying a wide range of insecure code patterns, it is not comprehensive and may miss nuanced or context-dependent vulnerabilities. Its detection efficacy was evaluated in CyberSecEval3. [It] achieved a precision of 96% and a recall of 79% in identifying insecure code.
The combined use of PromptGuard and AlignmentCheck improves performance on the AgentDojo benchmark. Moreover, Meta’s researchers suggest that this combination could yield even better results in more diverse or generalized adversarial scenarios beyond those covered by AgentDojo.
Meta’s researchers describe two workflows showcasing how LlamaFirewall can be integrated into agentic systems. In the first scenario, a travel planning agent uses PromptGuard to scan web content (such as travel reviews) for jailbreak-style phrasing and discards any suspicious pages. At the same time, AlignmentCheck monitors the agent’s token stream to detect if the goal shifts away from travel planning, in which case it halts execution.
In the second scenario, a coding agent generates SQL code based on developers’ input. The agent retrieves examples from the Web and checks it using CodeShield until it finds a correct solution.
Work on LlamaFirewall will continue in several directions, including support for multimodal agents, reduced latency, expanded threat coverage, and more realistic benchmarking.
