A new training technique lets smaller vision-language models outperform much larger ones on the unglamorous but commercially important task of reading long documents.
Researchers built a synthetic data pipeline that teaches models to reason over long visual documents — think hundred-page PDFs, legal filings, or scientific reports. The pipeline scores each page for relevance to a given question, extracts text evidence, and orders it by importance. Models trained on the resulting reasoning traces, using supervised fine-tuning gated by a control token, then have that reasoning ability merged into their weights at low strength — a process the authors call "internalized reasoning." Applied to Qwen3 VL 32B, the approach hit 58.3 on the MMLongBenchDoc benchmark, edging out Qwen3 VL 235B A22B, a model seven times larger, which scored 57.0. Applied to Mistral Small 3.1 24B, internalized reasoning beat distillation from a dedicated "Thinking" model variant by 3.8 points, while generating 12.4 times fewer output tokens than explicit chain-of-thought reasoning.
The token-efficiency finding matters most. Explicit reasoning chains are expensive to run at scale — enterprise document workflows process thousands of pages daily, and verbose outputs compound fast. A model that thinks internally and stays quiet is far cheaper to deploy. The benchmark result also challenges the assumption that bigger models are the default answer for complex document tasks.
The pipeline and trained models are being released publicly, which puts capable long-document reasoning within reach of teams that cannot afford to run 200-billion-parameter models. Whether the gains hold on messier real-world documents — scanned contracts, mixed-layout filings — rather than clean benchmarks remains the open question.