Anthropic, OpenAI, and other leading AI labs are reportedly war gaming what happens if a rogue AI system wreaks havoc on critical infrastructure, according to Axios.
Leadership at these companies has been privately running scenarios covering disruptions to financial services, telecom and internet systems, and utilities like power and water, Axios reports. The setup echoes how the Pentagon simulates nuclear exchange outcomes, and routine business-continuity planning is common in corporate America. What's unusual is the subject matter: many industry experts reportedly expect a major AI incident within the next six to 12 months. OpenAI told Axios its preparedness exercises are meant to ready teams for a range of outcomes, not predict an inevitable one.
Part of the planning reportedly centers on how to brief Congress after an incident, since the White House has so far opted for voluntary self-policing over binding AI rules. That detail matters more than the scenario planning itself, since it suggests labs expect to need political cover, not just technical fixes, when something goes wrong. Anthropic's own IPO prospectus devotes nearly a third of its risk factors section to existential risks to humanity, an unusual admission for a company simultaneously asking investors for money.
Aviation and nuclear power only built real safety cultures after disasters forced the issue, a former OpenAI safety staffer has noted, and these labs are trying to get ahead of that curve while still pouring over $1 trillion into the infrastructure they say could go wrong.