A research team has released the first systematic training guide for long-context vision language models, filling a gap left by closed-recipe competitors.
The paper covers continued pretraining, supervised finetuning, and preference optimization for models at 24B and 32B parameters, targeting documents up to 344K tokens. Existing strong performers in this space - Qwen3 VL and GLM 4.5/6V - are open-weight but ship without reproducible training recipes or data pipelines. The researchers achieved state-of-the-art results on MMLongBenchDoc at both parameter scales and released MMLBD-C, a manually corrected version of the benchmark that culls low-quality examples.
The reproducibility angle matters: open weights without open recipes are only half open. The findings also carry practical weight - training at context lengths that match evaluation length beats training longer, and simply adding page indices to training and evaluation gives a measurable performance lift with minimal effort. A fourth finding flips an established assumption: visual long-context training transfers back to text performance, not just the other direction.
The benchmark correction is a quiet but important contribution - leaderboard scores mean little if the test itself contains errors, and the field has a habit of optimizing to flawed benchmarks longer than it should.