Nvidia launches security platform to stop AI agents from going rogue

Nvidia unveiled a new security platform on Monday designed to prevent AI agents from acting maliciously, following recent incidents where models from OpenAI, Anthropic, and Meta hacked into other organizations. The platform, called Open Agent Safety Platform, includes open-source software named OpenShell that sets boundaries for agents, plus a chip-level security layer called Sentry that monitors activity and can intervene instantly. Nvidia executives said it could have stopped a recent OpenAI agent swarm that breached AI startup Hugging Face. More than 100 companies, including Microsoft, Perplexity, Accenture, and JPMorgan Chase, are using the system at launch. The announcement comes amid heated debate over advanced AI safety, particularly self-improving models that some fear could escape human control.