A new technique called KV-Rescue uses a tiny, full-context model to patch the memory lost when large reasoning models trim their own history to save space.
Researchers found that when a reasoning model's KV-cache - its memory of prior reasoning steps - gets aggressively trimmed to cut costs, the resulting errors look less like the model running out of ability and more like it's missing context it once had. They tested this by comparing a 7B model with a trimmed cache against a much smaller 1.5B model that kept its full memory intact; the two made different kinds of mistakes, and simply picking the better answer between them recovered 79% of the accuracy the trimmed model had lost. KV-Rescue builds on that finding: it interleaves reasoning steps from a full-context helper model with those from the memory-trimmed model, and uses an online detector to cut the trimmed model off early if its output starts to ramble or repeat. Tested across five math benchmarks on both the 7B and 72B versions of Qwen2.5-Math at an aggressive eviction budget, the method recovered an average of 87% of the accuracy lost to trimming, while cutting the trimmed model's token generation by 43%.
KV-cache eviction is one of the main levers teams pull to keep long reasoning chains affordable, but it's historically been a blunt instrument - you trade accuracy for memory and hope the drop is small. KV-Rescue needs no retraining, and it suggests that gap is mostly about missing information rather than missing capability, meaning a cheap helper model can plug most of it back in without keeping the full cache around.
It's a patch, not a fix - the eviction itself is still lossy, and running two models at once has its own overhead, however much shorter it makes the losing model's rambling.