New Agent Data Injection Attack Can Make AI Agents Misclick or Run Attacker Commands
Researchers from Seoul National University, the University of Illinois Urbana-Champaign, and Largosoft have discovered a new class of attack called agent data injection (ADI), which can make AI agents misclick or run attacker commands. This attack corrupts the facts an ag…
Intelligence analysis by Llama
The attack, known as probabilistic delimiter injection, exploits the way language models read punctuation by guesswork. An attacker can sprinkle punctuation-like characters into a field they control, making the model read them as real structure that was never there. This can lead to AI agents clicking the wrong buttons, running malicious commands, or approving fake code changes.
Imagine you're asking a friend to get you a sandwich. But someone sneaks in and changes the order to get you a pizza instead. That's basically what's happening with this new attack on AI agents. The attacker is tricking the agent into doing something it wasn't supposed to do, like clicking the wrong button or running a malicious command.
Analysis
A New Class of Attack: Agent Data Injection (ADI)
The researchers from Seoul National University, the University of Illinois Urbana-Champaign, and Largosoft have identified a new class of attack that can make AI agents misclick or run attacker commands. This attack, known as agent data injection (ADI), corrupts the facts an agent trusts, allowing it to carry out the attacker's instructions without hijacking the agent's task.
How ADI Works
The attack exploits the way language models read punctuation by guesswork. An attacker can sprinkle punctuation-like characters into a field they control, making the model read them as real structure that was never there. This can lead to AI agents clicking the wrong buttons, running malicious commands, or approving fake code changes.
Probabilistic Delimiter Injection
The method behind ADI is called probabilistic delimiter injection. Agents wrap their data in punctuation that marks where one piece ends, and the next begins: quotes and braces, tags, brackets, and line breaks. A normal program reads that punctuation by strict rules. A language model reads it by guesswork. So an attacker can sprinkle punctuation-like characters into a field they control, and the model will often read them as real structure that was never there.
Testing the Attack
The researchers built three working attacks on real, shipping tools: On web agents (Claude in Chrome, Google's Antigravity, and Nanobrowser), a planted product review reuses the ID of a real button. The agent means to click "Read More" and clicks "Buy Now" instead, placing an order the user never made. On coding assistants (Claude Code, OpenAI's Codex, and Google's Gemini CLI), a GitHub comment forges its author line to look like a project maintainer wrote it. Told to apply the maintainer's fix, the agent will run the attacker's command on the developer's machine if the developer approves what looks like a routine step.
Defenses Against ADI
The researchers found that adding a short random tag to field names roughly halved the attack's success rate, from about 49% to 29%. They also found that stripping the punctuation out cut the attack down too, but it broke the agents' ability to read normal things like links and file paths along with it. The researchers describe proof-of-concept attacks only, and there is no public report of ADI being used in the wild.
Key points
- Researchers have discovered a new class of attack called agent data injection (ADI) that can make AI agents misclick or run attacker commands.
- The attack exploits the way language models read punctuation by guesswork.
- Adding a short random tag to field names can roughly halve the attack's success rate.
- Stripping punctuation from data can also cut the attack down, but it breaks the agents' ability to read normal things like links and file paths.
The discovery of this attack highlights the need for more robust defenses against AI agent manipulation. Researchers and developers are already working on solutions, such as adding random tags to field names or stripping punctuation from data. These efforts could lead to more secure AI agents and a safer online environment.
If left unchecked, this attack could lead to widespread misuse of AI agents, compromising the integrity of sensitive data and systems. The attackers could use this vulnerability to steal sensitive information, disrupt critical infrastructure, or even manipulate public opinion.
