Telling an AI grader to never award partial credit can make it stop grading altogether.
Researchers tested large language models as graders on two real college exams: a 570-student Computer Vision exam graded across 171 configurations, and a 1,038-student Machine Learning exam graded across 162 more. On the first exam, the best model hit a mean absolute error of 1.64 out of 35 points, beating the 2.61-point disagreement between two human graders on the same students. But adding a short strict-grader preamble that told models never to give partial credit pushed 14 of 17 open-weights models into error rates of 8 points or worse, and three stopped grading entirely. Closed flagship models from three vendors survived the same instruction, just shifting their scoring calibration.
The second exam showed the effect is not universal: the same harsh instruction broke ten more models but actually improved seven that had been grading too generously under neutral prompts. That is the uncomfortable part for anyone hoping to automate grading. One phrase in a prompt can help or wreck a model's judgment, and which one it does depends on the exam, not the model.
Light fine-tuning fixed most of it: training a single adapter on roughly 3,900 pooled graded answers brought five small open models to parity with human grader agreement and nearly erased their sensitivity to harsh phrasing, evidence that a general-purpose chatbot is a poor substitute for a purpose-built grader.