AI models can now lay out a scientific poster — but can they do it without making things up?
Researchers have released PosterHarness, an open evaluation harness that reframes scientific poster generation as a measurable instruction-following problem. The system works by imposing a "placeholder-first" contract: the model handles visual design — typography, color, reading flow, background — but is explicitly barred from drawing any data-bearing figures. Every chart or graph region must be left as a labeled empty box; a deterministic compositor then drops in the real figures from the source paper at the correct coordinates. That separation makes failures auditable. Placeholder accuracy, aspect-ratio compliance, and whether the model snuck in a synthesized graphic are all logged as explicit rejections rather than buried inside plausible-looking output. A pilot run across 12 papers — split between high-energy physics and AI research — showed the contract cut VLM-generated fake figures from 34 to zero across three test papers.
The benchmark matters because existing evaluations for multimodal models focus heavily on aesthetics or factual question-answering, not on whether a model fabricated a figure it had no business drawing. Scientific communication carries a higher trust bar than a marketing slide, and a hallucinated chart in a conference poster could quietly spread before anyone notices the source paper never contained it.
For comparison, PosterHarness trades some information retention and speed against Paper2Poster, the closest prior system, in exchange for higher-resolution output and stronger visual preference scores in blind evaluations — a trade-off the authors characterize as a difference in regime, not a win. All prompts, audit scripts, and artifacts are public, which is the right call: a benchmark nobody can inspect is just a leaderboard.