A new paper argues AI labs are showing the same organizational warning signs that preceded some of history's worst engineering disasters.
The paper, posted to arXiv, compares present-day AI development to the Space Shuttle Challenger explosion, the Three Mile Island nuclear accident, and the Boeing 737 MAX crashes. It identifies structural mechanisms common to all three failures and argues those same mechanisms are present in how AI organizations operate today. Its central claim is that safety processes can be followed in full and still fail to prevent catastrophe, because the process itself stops matching the risks it was designed for. The author frames this as an ongoing, interruptible situation rather than a foregone conclusion.
Most AI risk debate obsesses over whether models get too capable or too autonomous. This paper's angle is different: it says the organizations building those models face the same slow drift toward failure that doomed NASA, the nuclear industry, and Boeing, regardless of how the technology itself performs. That reframes safety audits and compliance checklists as necessary but not sufficient.
Aerospace learned the hard way that passing every checklist is not the same as being safe; nothing about training on GPUs instead of building rockets changes that math.