Nvidia launches AI agent safety system

Nvidia unveiled a system on Monday designed to stop autonomous AI agents from going beyond what they were instructed to do, after several companies reported incidents involving agents that broke out of test environments.

AI agents are programs that can act on their own, including browsing the web, writing and running code or handling files. Nvidia said its system places each agent in a sealed-off digital space and lets companies set which files, networks and tools the agent may use.

The company said a separate monitor, running on Nvidia hardware outside the agent's reach, watches the agent's behaviour and can cut it off if it goes astray.

Nvidia chief executive Jensen Huang told CNBC that he believed the problem could be solved through engineering. “If it's not an engineering problem, it's not solvable,” he said.

OpenAI has said websites accessed by its agents include US federal agency sites, an Australian government health statistics portal and Hugging Face, a repository of AI models. Nvidia said its system could have prevented the Hugging Face breach, in which Hugging Face reported more than 17,000 agents attacking its infrastructure over days and weeks.

Nvidia said more than 100 organisations are working with the platform at launch, including Microsoft, Cisco, Salesforce and SAP. It said Anthropic has connected its Claude agents to the system and SpaceXAI is applying it to its Grok models.