Language models make data referencing errors in tables even when they understand the table's structure — and a new study is the first to measure the problem at scale.
Researchers evaluated models ranging from 1.7B to 20B parameters across multiple table tasks and found that all of them misquote or omit cell values during intermediate reasoning steps, not just in final answers. That distinction matters: a model can arrive at the right conclusion through a chain of quietly wrong citations, which erodes trust in how it got there. The team then trained a lightweight 4B-parameter critic model that detects these errors with an average F1 score of 78.2%, including on table types it was not trained on. Plugging that critic into inference pipelines — via filtering and rejection sampling — lifted answer accuracy by up to 12.0%.
The finding reframes what "table understanding" actually means. Passing table benchmarks has become a standard capability claim for frontier models, but those benchmarks typically score final outputs, not the reasoning chain. A model that lands the right answer while hallucinating three intermediate cell values is not, in any useful sense, reading the table carefully.
The 4B critic outperforming naive scaling is the understated result here: rather than training a bigger model to make fewer mistakes, a small, cheap watchdog catches what the larger model misses — a pattern that has shown up in code review and factuality checking, and is quietly becoming a design pattern worth watching.