Switching the language an AI negotiates in can matter more than swapping the model itself.
Researchers ran controlled multi-agent simulations pitting large language models against each other across three negotiation game types: Ultimatum, Buy-Sell, and Resource Exchange. They held game rules, model weights, and incentives constant, then varied only the language — testing English against four Indic languages: Hindi, Punjabi, Gujarati, and Marwadi. The results showed that language choice alone could reverse proposer advantages and shift how surplus was divided between parties. The effects weren't uniform: Indic languages made distributive negotiations less stable but produced richer, more exploratory behavior in integrative settings where both sides could theoretically gain.
The implication is uncomfortable for anyone deploying AI in multilingual contexts. Nearly every published benchmark for LLM negotiation runs exclusively in English, which the authors argue yields incomplete — and potentially misleading — conclusions about how these models actually behave. If a model negotiates more aggressively or more cooperatively depending on whether it's prompted in Gujarati versus English, then English-only safety and capability evaluations are testing a different agent than the one users in South Asia will encounter.
The AI evaluation field has spent years arguing about which benchmarks matter; this paper suggests the language the benchmark is written in might matter just as much as which benchmark you pick.