A new technique splits one big language model into a team of smaller ones that cross-check each other's reasoning.
The approach, called collaborative learning, replaces a single large language model with several smaller, differently-configured models that share roughly the same total parameter budget, a step the researchers term model slicing. Because each smaller model is built differently, it can catch reasoning errors that would otherwise slip past a single model talking only to itself. The team adds pyramid supervision, which pairs distilled interaction protocols with step-by-step feedback to train the group on when to hand off a problem and when to challenge a collaborator's answer. They tested the setup across four task types: multi-hop question answering, math reasoning, code generation, and open-ended constrained generation.
The trained group of smaller models matched or beat both the original large model and existing multi-agent baselines, including reinforcement-learning-based ones, while running more efficiently. That efficiency claim matters because most multi-agent LLM setups trade extra reliability for extra compute and latency; this one claims to get the reliability without the usual overhead.
Multi-agent LLM pitches are common these days, and most lean on brute-force redundancy rather than genuine diversity of reasoning. Whether model slicing holds up outside this paper's own benchmarks is the question worth asking before anyone builds a product on it.