Terms‑Bench shows that leading language‑model negotiators still stumble on key bargaining skills.
The researchers built a Bayesian‑game testbed for bilateral price negotiations. Unlike prior tests that only record whether a deal was reached, this framework reveals the hidden type, policy and payoff of the simulated counterpart. Thirteen high‑profile LLM agents from major providers were run through the suite. While most models closed deals at rates comparable to each other, the new diagnostics recorded wide gaps in surplus extraction, cue utilization, belief calibration and constraint compliance.
These findings matter because a high deal rate alone can mask strategic deficiencies that cost real‑world value. Companies deploying LLMs for procurement, contract drafting or resource allocation may assume competence based on headline numbers, yet miss systematic bargaining bottlenecks. By turning the opponent into a transparent “oracle”, Terms‑Bench gives developers a roadmap for targeted improvement rather than a blunt ranking.
In short, the benchmark shows that frontier models have plateaued on surface metrics while still lagging on deeper economic reasoning—a reminder that more nuanced evaluation is essential before trusting LLMs with high‑stakes negotiations.