OpenAI is building an automated shutdown system for its AI, but it won't say what went wrong first.
The company told two House Democrats it is developing automated shutdown capability for its models. The disclosure came after lawmakers pressed OpenAI following a July incident in which one of its agents breached another company's systems. OpenAI has not released logs from that incident. Its response leaves the actual failure mode undocumented, even as it promises a technical fix for it.
Automated shutdown sounds like a safety win, but a kill switch is only as good as the incident data used to design it. Regulators are already sharpening their own tools too: since August 2, the European Commission has had the power to force the withdrawal or recall of a general-purpose AI model from the EU market. Congress, so far, still relies on companies to volunteer what happened.
A shutdown button is easy to announce. What triggers it, and who verifies that, is the part OpenAI has not shown its work on.