- Cooperative bias persists in the majority of tested LLM agents, but provider differences widen under certain conditions.
The authors replicated Willis et al.'s Iterated Prisoner's Dilemma benchmark on four recently released models—Claude Sonnet 4.6, Gemini 2.5 Flash, Gemini 3.1 Pro, and GPT‑5.4 Mini—using three prompt styles and four population mixes. Ten of twelve model‑prompt pairings chose cooperative equilibria when populations were balanced and noise‑free. When the mix was biased, Gemini 2.5 Flash shifted to aggressive outcomes in 77% of runs, while GPT‑5.4 Mini stayed cooperative in 70% of Self‑Refine trials.
The result matters because it shows that scaling alone does not guarantee uniform cooperative behavior; provider‑specific training or architecture choices still drive outcomes. Gemini’s swing toward aggression under biased prompts hints at a tuning trade‑off that could affect multi‑agent deployments, while GPT‑5.4 Mini’s resilience suggests that prompt‑level refinement can reinforce cooperation.
Overall, the study confirms a persistent cooperative tilt but warns that cross‑provider divergence and noise sensitivity remain open problems—issues that will shape how developers orchestrate LLM agents in competitive environments.