Fine-tuning large language models on new facts does not guarantee those facts show up when the model needs to reason.
Researchers studying this problem have formalized it as the "Knowing-Using Gap": models can accurately recall injected facts in isolation yet fail to apply them in downstream tasks. The gap has two signatures — an accuracy difference between raw recall and applied reasoning, and a time lag where generalization trails memorization even as training continues. To probe the mechanism, the team developed a technique called self-patching, which relocates internal activations at inference time to identify exactly where the model's wiring breaks down. Their finding: memorized representations get stored internally but are not routed to the layers that actually drive computation during reasoning.
This matters because fine-tuning is the dominant method organizations use to inject domain knowledge into off-the-shelf models — and most evaluation benchmarks test recall, not applied reasoning. A model that aces a knowledge quiz but fumbles the downstream task is not actually more capable; it just looks like it is. The routing failure hypothesis, if it holds up under wider scrutiny, reframes the problem from "how much data does the model need" to "where does the data need to land."
The team also showed a practical payoff: a simple heuristic strategy built on their diagnostic recovered 58-75% of the gap between failure and oracle performance, which is a meaningful result for a heuristic, though the range is wide enough to warrant skepticism about how consistently it generalizes outside their test conditions.