Researchers have built a negotiation benchmark that makes "did a deal happen?" look like the wrong question.
A team introduced TERMS-Bench, short for Testbed for Economic Reasoning in Multi-turn Strategy, a framework designed to expose exactly where AI bargaining agents fail and why. Unlike existing evaluations that pit LLMs against each other and count successful closures, TERMS-Bench uses a Bayesian-game setup: an agent negotiates against a simulated counterpart whose hidden preferences, policies, and payoff structure are visible to the evaluator but not the agent. That turns the opponent from a black box into a diagnostic instrument. The researchers ran 13 LLM agents from major providers through bilateral price negotiations, measuring not just deal closure but surplus extraction, contextual cue use, belief calibration, and compliance.
The headline finding is that frontier models are converging on deal rate while diverging significantly on everything else that determines whether a deal was actually good. For anyone deploying an AI system to handle procurement, vendor contracts, or hiring negotiations, that gap matters: your agent may be closing on terms a more capable model would have improved. The framework also enables agent-specific failure analysis, letting developers pinpoint whether a model struggled because it misread signals, held miscalibrated beliefs, or simply conceded too fast.
Deal rate as the north star for negotiation AI is a bit like grading a poker player on whether they stayed in the hand. The point was never just to play.