A Malicious Webpage Could Poison Your Local AI Model Behind NVIDIA NemoClaw
A vulnerability in NVIDIA NemoClaw allows an attacker to take unauthenticated control of the local Ollama instance serving an AI agent and plant hidden instructions inside the model itself.
Intelligence analysis by Llama
A malicious webpage can poison a local AI model by exploiting a weakness in NVIDIA NemoClaw, allowing an attacker to take control of the model and inject hidden instructions.
Imagine you have a computer program that can understand and respond to human language. An attacker could create a special webpage that tricks the program into doing something bad, like revealing secret information. This is like a computer virus that can spread through the program's own language.
Analysis
NemoClaw and Ollama Vulnerability
A recent report by Oasis Security has disclosed a weakness in NVIDIA NemoClaw that could allow an attacker to take unauthenticated control of the local Ollama instance serving an AI agent and plant hidden instructions inside the model itself. The vulnerability is caused by the fact that NemoClaw starts Ollama with OLLAMA_HOST=0.0.0.0:11434, binding the model server to every network interface. This allows an attacker to modify the model's chat template so that hidden instructions are applied to every later conversation.
How the Attack Works
The attack works by exploiting the fact that the API on port 11434 has no authentication and relies on two middleware layers to block browser-originated requests. When the bind address is not loopback, the Host header check is skipped entirely. The Cross-Origin Resource Sharing (CORS) layer then treats the request as same-origin and allows it, because the Origin and Host headers both carry the attacker's own domain. The attacker can then use DNS rebinding to close the gap and make the request appear as if it is coming from the same origin.
Fixing the Vulnerability
The fix for this vulnerability is to validate the Host header on the server side to allow only a set of authorized values. This can be done by setting the Host header to a specific value in the NemoClaw configuration file. Additionally, the NemoClaw proxy can be configured to refuse to start against a backend that is not bound to loopback. This can be done by setting the OLLAMA_HOST environment variable to 127.0.0.1:${port} on the Ollama systemd unit or by setting the NEMOCLAW_OLLAMA_PROXY_SKIP_BIND_PROBE environment variable to 1.
Implications of the Vulnerability
The implications of this vulnerability are significant. If an attacker is able to take control of a local AI model, they may be able to inject hidden instructions that are executed during inference. This could potentially allow the attacker to access sensitive information or take control of the model's behavior. The vulnerability highlights the importance of securing local AI models and the potential risks of unauthenticated access to sensitive information.
Key points
- A vulnerability in NVIDIA NemoClaw allows an attacker to take unauthenticated control of the local Ollama instance serving an AI agent.
- The vulnerability is caused by the fact that NemoClaw starts Ollama with OLLAMA_HOST=0.0.0.0:11434, binding the model server to every network interface.
- The attack works by exploiting the fact that the API on port 11434 has no authentication and relies on two middleware layers to block browser-originated requests.
- The fix for this vulnerability is to validate the Host header on the server side to allow only a set of authorized values.
If this vulnerability is addressed, it could lead to improved security measures for local AI models, making them more resistant to attacks and protecting sensitive information.
If left unaddressed, this vulnerability could lead to widespread exploitation, allowing attackers to inject hidden instructions into AI models and potentially access sensitive information.
