Researchers can stop Gemma 4 from cycling through lists indefinitely by editing as few as one neuron - though the same technique cannot fix a model that simply does not know a fact.
Google's Gemma 4 instruction-tuned models have a documented and reproducible failure: ask them to enumerate a long factual list - every episode of a TV series, all 88 IAU constellations, the original 151 Pokemon - and they collapse into repetition at rates as high as 95%. The behavior survives prompt rewording, inference-engine changes, and most sampling adjustments, which rules out easy workarounds. Researchers used per-layer ablation and per-neuron attribution to trace the loops to a small cluster of MLP neurons, or in the 26B-A4B mixture-of-experts variant, a handful of routed experts, then suppressed them with static weight edits. In the smallest model, the fix required inverting the sign of exactly one parameter.
This is one of the cleaner demonstrations of what mechanistic interpretability research has been promising: that specific, harmful behaviors in large models can be localized to specific, small sets of parameters and removed without collateral damage. General-purpose benchmark scores held steady after the edits, suggesting the surgery was targeted rather than blunt. For teams building on top of Gemma 4 and hitting these loops in production, weight edits now appear to be a viable mitigation rather than a model-swap.
The paper is careful about what the approach cannot do. The harder failure - what the researchers call "doom looping," where the model self-corrects in circles over a fact it cannot recall - survives the neuron edits. Weight surgery can delete a loop. It cannot supply a missing fact.