AI Executives Warn of Extinction Risk, Sparking Calls for Development Slowdown

Key points

• Jacob Coxon resigned from Anthropic on September 8, 2026, stating AI could kill humanity by the end of the decade.
• Evan Hubinger, head of Anthropic’s alignment science, estimated a greater than 10% risk of human extinction within ten years.
• Dario Amodei, Sam Altman, and Elon Musk publicly supported calls to slow AI development, though not halt it entirely.
• The Center for AI Safety categorizes risks into rogue AIs, malicious use, race dynamics, and organizational failures.
• Critics argue that fears of extinction distract from immediate issues like data center pollution, labor exploitation, and AI-driven disinformation.

Background

This debate follows recent US actions to accelerate AI growth, including President Trump’s plans for an 'AI Force' and an 'AI Czar' in September 2026. It also occurs amid tensions over Chinese firms allegedly distilling US AI models, highlighting the competitive pressure that may drive rapid development despite safety concerns.

How outlets are covering it

Nature and NBC News emphasize the viral nature of the warnings and the consensus among major AI leaders to slow development. The Guardian’s Toby Walsh outlines five specific extinction scenarios, from bioweapons to nuclear war, but notes that many are unlikely due to real-world friction. NBC News highlights immediate risks like AI-assisted bioweapons and military integration, while critics like Timnit Gebru and Keegan McBride dismiss extinction fears as 'fear-mongering' that obscures more tangible harms such as labor displacement and environmental damage.

Why it matters

The warnings from top AI executives signal a potential shift in industry priorities toward safety and regulation. If legislators act on these concerns, it could slow the pace of AI development, affecting economic growth, military capabilities, and global competition, particularly with China.

What to watch

Lawmakers, including Senator Bernie Sanders, are considering bills to ban 'artificial superintelligence.' California Governor Gavin Newsom has ordered a task force to develop an 'AI kill switch.' The industry is also exploring recursive self-improvement, which could accelerate the timeline for advanced AI capabilities.