A new study finds that knowing what a few neighbors think helps predict what an entire simulated society of AI agents will believe, but only sometimes.
The paper, "Local Predictability and Collective Fidelity in LLM-Agent Societies" (arXiv:2609.35813), asks whether cheaper stand-in models can replace full LLM-agent simulations without losing accuracy. Researchers compared individual-level predictions against collective forecasts using 9,455 previously published simulation trajectories, plus new experiments on opinion dynamics. Giving an agent information about its neighbors improved individual prediction accuracy across all 16 public datasets tested, and also improved pooled forecasts on held-out questions, though the size of that collective benefit depended on how the model was transferred to new settings. In a follow-up test with 24 new statements, the team could not reproduce an earlier finding about how conversation history affects predictions from a starting state, though the Qwen model still benefited from history once it had observed at least three rounds.
Multi-agent LLM simulations are increasingly used to model social media dynamics, opinion spread, and market behavior, and swapping in cheaper surrogate models is the obvious way to make them affordable at scale. This paper is a reality check: getting individual predictions right does not reliably translate into a simulation that reproduces group-level outcomes, which is the number that actually matters for a simulation to be useful. The authors recommend validating collective results directly, capping how much information agents can observe, and testing against simple baselines, an implicit critique of how much current work skips that step.
It is a useful corrective in a field that has moved fast since 2023's "generative agents" demos went viral: individually convincing chatbots can still add up to a crowd that behaves nothing like a real one.