Accuracy alone is a bad report card for AI that keeps learning.
Researchers studying multimodal large language models — systems that reason across text, images, charts, OCR, and documents — found a failure mode that standard benchmarks miss entirely. When these models are updated on new tasks, they can preserve their answer accuracy while silently shifting to different or weaker evidence sources. The team calls it "hidden evidence-use forgetting": the model still says the right thing, but for the wrong reasons. To address it, they built RCL, a continual learning framework that freezes a prior checkpoint as a behavioral reference and penalizes the model for drifting in how it weighs evidence channels — without requiring it to store and replay old training data.
This matters because the AI industry's standard measure of catastrophic forgetting — did the model's accuracy on old tasks drop? — turns out to be an incomplete test. A model that shifts from reading a chart to guessing from context might score identically on a benchmark while becoming quietly unreliable in production. RCL showed improvements in both conventional forgetting metrics and this new grounding-drift measure across four evaluation benchmarks.
The finding lands at an awkward time for labs racing to ship continuously updated multimodal assistants. If the standard eval passes a model that has lost its evidentiary footing, the safety and reliability case for those systems is weaker than the leaderboards suggest.