AI/ multi-agent-rl · reinforcement-learning · ai-research · kl-divergence

AI Agents Learn to Talk Only When They Disagree

A new belief divergence threshold cuts chatter between RL agents, beating a rival method on harder tasks but losing on easier ones.

A researcher has built a way for AI agents to shut up until they actually disagree.

The paper proposes a gating rule for multi-agent reinforcement learning: instead of chattering every timestep, or learning when to speak through a noisy reward signal the way IC3Net does, each agent keeps a running belief about the state of the world, computed from its own memory network. Agents only send a message when the gap between their beliefs, measured with KL divergence (a standard way to score how far apart two probability distributions are), crosses a set threshold. The method was tested on the Predator-Prey coordination benchmark from IC3Net at two grid sizes, and on the MPE simple_spread task, across five seeds. Results were split: on the smaller 10x10 grid, IC3Net's own gating beat the belief-based approach at every threshold tried, but on the harder 20x20 grid, a threshold of 0.5 won out, cutting average steps and lifting success rate from 31 percent to 42 percent.

That the win depends on task difficulty is the real finding here, not a clean sweep. It suggests threshold-based gating only pays off once coordination gets hard enough that wasted, noisy messages start actively hurting agents, a useful data point for anyone building AI agent swarms that need to ration bandwidth along with decisions. The side effect, richer belief representations that lifted MPE rewards by 12 points and cut variance 26-fold even with gating switched off, may end up being the more durable contribution.

It's a single-author preprint with results that flip depending on map size, so treat this as an interesting knob to test, not a new default for multi-agent systems.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →