AI research tools that score as well as human annotators may still be measuring the wrong thing entirely.
A paper posted to arXiv argues that when a large language model codes a construct in text — say, labeling a sentence as expressing hostility — matching a human annotator's verdict only proves the model is reliable, not valid. The model might be latching onto a surface correlate that happens to align with the label without actually tracking the underlying theoretical construct. The authors call this a construct validity gap, and point out that no standard method currently catches it. To fix that, they propose grain calibration: break a construct into clause-level components, test each one against the source text with extractive evidence, and combine results through an explicit theory-derived rule rather than a single opaque model pass.
This matters because social science and computational linguistics have increasingly leaned on LLMs as cheap, fast replacements for human annotators. If those models are reaching correct codes through the wrong reasoning, entire bodies of research built on that output inherit the flaw — quietly. Grain calibration makes the reasoning structure visible, so a reviewer can see which components drove a code and where the model went wrong when it did.
The timing is notable. As LLM-based annotation pipelines move from academic experiments into production research tools and content moderation systems, the gap between "agrees with a human" and "measures what we think it measures" could carry real consequences. Agreement scores were always a proxy for validity; this paper argues it is past time to stop treating them as the same thing.