An AI model just sat the licensing exam that actually decides who gets to run a nuclear reactor - and it needed serious coaching to scrape by.
Researchers put Gemma 4, an open-weight 31-billion-parameter model, through the U.S. Nuclear Regulatory Commission's Reactor Operator Generic Fundamentals Examination, using every March sitting from 2015 to 2021: 14 papers, seven for pressurized-water reactors and seven for boiling-water reactors, 697 questions total. They graded it exactly as regulators grade human candidates - 80% to pass, no rounding. Straight out of the box the model scored 51.94% and passed zero papers. After supervised fine-tuning on distilled chain-of-thought reasoning, combined with retrieval over the Department of Energy's Fundamentals Handbooks, it passed 8 of 14 papers, reaching 80.23% on pressurized-water-reactor material and 79.77% pooled across both reactor types.
The more interesting finding isn't the pass rate, it's what didn't work. A fancier retrieval method built specifically for this kind of fine-tuning, RAFT, consistently scored 2.2 to 2.3 points worse than plain fine-tuning, across every reactor type and chunking setup tested. And the best way to chop up reference documents for retrieval flipped once the model was fine-tuned, so a chunking strategy tuned for the base model actively worked against its fine-tuned version. Bigger context windows and smarter retrieval pipelines aren't automatically better; they have to be re-tuned for whatever model state is actually running.
Even the passing score came with an asterisk: the statistical confidence interval around 79.77% straddled the 80% cutoff, meaning the model's pass is closer to a coin flip than a certification. Call it exam-crammed, not qualified.