California Rep. Ro Khanna is drafting a bill to ban self-improving AI models until Congress sets safety rules for them.
Khanna shared a summary of the Human Control Over AI Act first with CNBC, according to reporting published Monday. The bill would prohibit AI systems that can modify or improve themselves without human oversight, pending federal safeguards from lawmakers. Khanna, a Democrat, described the stakes as civilizational. The bill's full text has not been made public yet.
The idea targets a real technical trend: leading AI labs have spent the past two years chasing systems that can recursively improve their own training, reasoning, or code. Congress has mostly regulated AI after the fact, through disclosure rules and sector-specific guidance, rather than restricting a specific technical capability before deployment. A ban on self-improving models would be one of the first attempts to draw a regulatory line at the design stage instead of the output stage.
Whether the bill survives contact with a divided Congress is a separate question from whether it is written well, and right now the public only has a summary, not the actual text, to judge either one.