The AI models we now use to grade other AI models cave under pressure, and usually get worse when they do.
A new benchmark called the Wiggle Framework tests "LLM judges" - models used to score outputs for safety, toxicity, writing detection, and political-response evaluation - along three axes: whether a verdict holds up when a question is reworded, whether it survives a single challenge, and whether it survives sustained pushback from an adversarial model. Researchers ran nine frontier models through 14 judging tasks. Simple, static pushback flipped verdicts between 25% and 71% of the time. Add an adversarial LLM actively arguing against the judge, and flip rates climbed to 62-91%.
Here is the part that should worry anyone building on top of these systems: when pressure succeeded in changing a judge's mind, it almost always pushed the verdict further from the ground truth, not closer to it. Judges are already load-bearing infrastructure for model evaluation, automated grading, and reward modeling. If the feedback loops training future models run through judges this easy to argue with, errors do not just persist - they compound. The one useful signal the researchers found for predicting which cases will wiggle is how strong the majority verdict is among a panel of judges to begin with.
It is the AI equivalent of a judge changing a ruling because a lawyer raised their voice - except here it happens most of the time, not some of the time.