AI agents must be treated as untrusted systems: Researchers
Researchers from Google and others say AI-agent security should be built into the whole system, not just the model, to reduce attacks.
Intelligence analysis by GPT-5.4 Mini
A new paper argues AI agents should be treated like untrusted parts of a larger computer system, with security controls around permissions, data flow, and instructions. The warning matters for crypto because AI agents are already being used to trade, manage wallets, and interact with protocols.
Researchers are saying an AI helper should not be trusted like a careful human. It should be treated more like a tool that can make mistakes if someone tricks it.
The safe way is to put fences around it. It should only do small jobs, keep secret things in locked boxes, and not be able to run wild with wallet money.
That matters in crypto because these helpers can trade coins or move funds. If the rules are weak, it is like giving a robot a house key and hoping it never opens the wrong door.
Analysis
What the researchers argue
A paper released on May 20 by researchers from Google, Gray Swan AI, EmbraceTheRed, and several universities says AI-agent security should be treated as a systems security problem. Their core point is that making the model itself more robust is not enough. The agent has to be surrounded by controls that assume it can be fooled.
The three controls they emphasize
The researchers say three mechanisms could eliminate a large share of attacks. First, agents should clearly separate instructions from untrusted data so malicious text cannot masquerade as commands. Second, agents should only get the minimum permissions needed for a task, rather than broad access. Third, the wider system, not the agent, should control where sensitive data can go.
Why this lands in crypto
The article ties the research to a growing crypto use case: AI agents that trade, launch tokens, and interact with Web3 services on behalf of users. That makes security failures more dangerous because the agent may touch wallets and protocols directly. Cointelegraph cites a recent case in which Bankr disabled transactions after identifying an attacker who had gained access to at least 14 wallets. The piece also quotes prior comments from Merkle Science and Sahara AI leaders, who argue that safe setup, sandboxing, approval gates, and limited permissions are essential before an AI agent should handle funds.
Key points
- Researchers from Google and partners say AI-agent security should be treated as a systems problem, not just a model problem.
- They argue agents need clear separation between instructions and untrusted data to reduce injection-style attacks.
- They recommend minimum permissions and system-level control over sensitive data flows.
- The article connects the warning to crypto use cases where AI agents can touch wallets, trades, and protocols.
- A recent Bankr incident is cited as an example of how agent-linked systems can become security risks.
