Large language models don't cooperate in the Prisoner's Dilemma because they've reasoned their way there. They cooperate because of the word "Cooperate."
Researchers ran Llama-3-70B, Qwen2.5-32B, Qwen2.5-72B, and the smaller Llama-3-8B through the classic Prisoner's Dilemma under direct prompting. The three larger models locked into full cooperation every time, with zero variance across repeated runs, the furthest possible result from the Nash equilibrium, the mathematically optimal move in the game. Llama-3-8B, by contrast, played close to Nash. Using a layer-by-layer technique called logit-lens analysis, the researchers found the models actually lean toward the Nash move for roughly three-quarters of their depth, only to get overridden by a late surge toward cooperation in the final layers.
That override tracks specific words, not strategic thought. Swap "Cooperate" and "Defect" for neutral labels and the effect disappears: across 48 payoff-randomized games with neutral wording, no model locked into cooperation or showed consistent equilibrium play. In the 8B model, researchers even isolated one exact point in the network where nudging a single value flipped the decision from cooperate to defect in a clean, predictable line. The effect is also contagious: drop one model that plays the Nash-optimal move into a mixed population, and cooperation collapses across the group.
Worth remembering next time a benchmark claims an LLM shows emergent cooperation or game-theoretic reasoning: it might just be reading the label on the button.