Language models know far more about cultural context than they use — and researchers have found the specific wiring responsible.
A study using mechanistic interpretability and the N4 cultural appropriation benchmark tested eight models across four architectures and found 2-3 mid-layer attention heads per model that causally drive what the researchers call "cultural binding" — the process of matching cultural items to the appropriate identity group. Knocking out connections on those heads reduced binding strength by 9-23%. The same heads appeared in both base and instruct versions of each model, pointing to pre-training as the origin, not fine-tuning. A scaling intervention at inference time, boosting those heads by a factor of 2-3, raised cultural differentiation accuracy by 1-3 percentage points without meaningfully degrading general reasoning.
The more striking finding is the gap between what models know and what they act on: knowledge probing showed models hold 3-5 times more relevant cultural information than they surface in outputs. That means the failure mode isn't missing data — it's a routing problem. Fixing it may not require retraining at all, just steering the right heads at generation time.
This fits a pattern emerging across mechanistic interpretability research: capabilities that look absent often turn out to be present but suppressed, gated by a small number of attention heads that researchers are only now learning to read. Whether the 1-3 pp accuracy gain holds outside benchmark conditions is the question that matters next.