Nvidia rolls out guardrails after rogue AI agents breach systems - KATU
High confidence: full text extraction produced 1970 characters.
Nvidia rolls out guardrails after rogue AI agents breach systems
(TNND) — Nvidia unveiled a new safety platform Monday aimed at keeping artificial intelligence agents within defined limits as companies give the technology greater autonomy.
The chipmaker said its Open Agent Safety Platform is designed to strengthen AI security from testing through deployment, providing controls across the software and hardware systems that run AI agents.
The launch follows a series of high-profile AI safety incidents, including OpenAI agents hacking into Hugging Face and the breach of an Australian health department website.
Nvidia executives said the platform could have prevented the Hugging Face breach, which involved a swarm of OpenAI agents acting autonomously.
From what we know, this new security platform could have stopped the breach if it was being used in frontier labs for model evaluation early on,” Justin Boitano, Nvidia’s vice president of enterprise AI, said in a media briefing.
The incidents have increased pressure on AI companies to show that increasingly autonomous systems can be safely controlled. Anthropic and OpenAI leaders have pushed for a coordinated slowdown in AI development to allow safety measures to catch up, while Nvidia CEO Jensen Huang has argued the risks can be addressed through engineering.
Nvidia said its open-source OpenShell software establishes a secure runtime boundary, tracing agent actions and enforcing limits on the systems and data they can access. The software can also be extended to work with third-party computing platforms, including Arm and Intel.
A separate tool, Nvidia Sentry, continuously monitors agent behavior and can quarantine an agent in milliseconds if it attempts to move beyond its assigned boundary, the company said.
More than 100 organizations are working with the platform’s technologies, Nvidia said, including Anthropic, Hugging Face, JPMorgan Chase, Microsoft, Perplexity, Salesforce and SpaceXAI.