A new position paper argues that AI training has been quietly erasing an inconvenient truth: people often disagree about what the right answer even is.
Most machine learning systems assume a single ground truth exists for every example. When human annotators disagree, the usual fix is to average their labels or vote for a majority, treating any leftover disagreement as noise to be cleaned up. The authors of this paper say that approach throws away real information. For many human-centered tasks, they argue, several interpretations of the same input can be simultaneously valid, and squashing them into one label hides the diversity of human perception and judgment that produced them in the first place. Their proposed fix is to model the full space of plausible interpretations rather than a single aggregated target, while still filtering out genuine annotation errors from meaningful disagreement.
This matters because the training-data pipeline is where a lot of AI's blind spots get built in, long before anyone talks about bias in the model itself. Tasks like content moderation, sentiment labeling, or facial-recognition judgments are exactly the places where reasonable people read the same input differently, and a model trained to predict one "correct" answer will confidently flag or clear things that a sizable chunk of users would call differently.
It is a position paper, not a shipped system, so there is no benchmark showing this approach beats aggregation in practice yet. But the diagnosis lines up with a persistent complaint about annotation pipelines: disagreement gets managed away rather than understood.