Large language models are being eyed for consequential ranking tasks — and a new paper asks whether they can actually be trusted to do it.
Researchers tested leading LLMs as pairwise judges in two high-stakes prioritization scenarios: emergency department triage and homelessness service allocation. Rather than asking a model to rank a large group at once — a task prone to error even for humans — the approach draws on social choice theory, eliciting comparisons between pairs and aggregating them into a full order. The problem is knowing, before you commit to that ranking, whether the model's judgments are consistent enough to act on. The paper proposes two diagnostics: a classical measure called the coefficient of consistency, which counts circular contradictions in the comparison graph, and inter-run variability metrics like Kendall's tau, which check whether the model produces similar rankings across separate runs.
The finding that matters: the three LLMs tested showed substantially different consistency profiles depending on the task, meaning a model that performs reliably in one domain may not in another. For practitioners considering AI-assisted triage or benefits allocation, that variability is not a minor technical detail — it is the difference between a defensible process and a liability.
AI is already drifting into high-stakes bureaucratic roles, often faster than the evaluation frameworks to govern it. This paper is a useful corrective: it does not argue LLMs cannot rank, only that the question of whether they should requires checking the consistency math first.