Nvidia unveils hardware-based safety platform amid AI safety debate

Key points

• Nvidia released the Open Agent Safety Platform on September 28, 2026, to help developers enforce safeguards for AI agents.
• The platform combines the open-source OpenShell runtime with NVIDIA Sentry on BlueField-4 DPUs to create a layered safety architecture.
• Jensen Huang recently criticized Anthropic and OpenAI for warning that AI development could threaten humanity, calling their stance 'odd' and urging them to focus on engineering solutions.
• The platform is built on five core principles, including verifiable policy, out-of-band enforcement, and controlling the path to the model to prevent agents from acting outside their intended scope.
• Nvidia argues that safety and security are part of AI acceleration, not obstacles to it, and that the industry should move 80% of compute into safety and alignment rather than just capability expansion.

Background

This development follows a series of public disputes between Nvidia and leading AI labs in September 2026. At Dreamforce, Anthropic CEO Dario Amodei called for a staged slowdown of AI development, while OpenAI CEO Sam Altman pressed for new security rigor following a hacking incident. Nvidia CEO Jensen Huang has consistently argued against existential risk predictions, stating that companies should simply stop releasing unsafe products rather than slowing down innovation. Huang has also been a key figure in the US government's AI-regulation debate, with President Trump turning to him for guidance on balancing growth and safety.

How outlets are covering it

Nvidia and its CEO Jensen Huang view AI safety as an engineering challenge that can be solved through rapid innovation and hardware-based controls, arguing that the industry should not slow down. In contrast, Anthropic and OpenAI have expressed concerns about the pace of AI development, with Anthropic's Dario Amodei calling for a staged slowdown and OpenAI's Sam Altman urging new security rigor. CNN frames the new platform as a tool to keep AI agents from 'going rogue,' while the Nvidia Developer blog emphasizes the technical details of the platform's layered safety architecture. The debate highlights a fundamental disagreement on whether AI safety should be achieved through regulation and slowdowns or through engineering and acceleration.

Why it matters

The launch of the Open Agent Safety Platform signals a shift in the AI industry's approach to safety, moving from a focus on model capabilities to a focus on securing autonomous agents. This could have significant implications for the development and deployment of AI in various sectors, as well as for the ongoing debate about AI regulation and the role of government in overseeing the industry. The platform's emphasis on hardware-based controls and real-time monitoring could set a new standard for AI safety, potentially influencing how other companies and governments approach the issue.

What to watch

Nvidia is inviting frontier labs, developers, and infrastructure providers to build with the Open Agent Safety Platform, suggesting that the company aims to establish it as a standard for AI safety. The platform's success will depend on its adoption by the AI community and its ability to address the concerns raised by Anthropic and OpenAI. The ongoing debate about AI safety and regulation is likely to continue, with Nvidia and other companies pushing for engineering-based solutions while others call for more caution and oversight.