No AI model tested so far can reliably replicate the way a human instructor marks up a student's essay.
Researchers released SEFORA, a public corpus of 564 student drafts paired with 8,240 inline annotations from real instructors, along with assignment prompts, rubrics, scores, and multi-draft revisions across several college writing genres. Alongside the dataset, they introduced UniMatch, an evaluation framework that breaks generated feedback into discrete units, scores each unit for semantic alignment against instructor-derived criteria, and uses optimal matching to produce precision, recall, and F1 scores. Across 74 experimental configurations — covering multiple large language models and settings — no configuration exceeded an F1 of 0.4. The researchers also found that performance gets worse, not better, as models generate more output.
The findings matter because writing feedback is one of the better-documented drivers of student learning, and the pitch for AI writing tutors depends on that feedback being good. Until now, the field lacked both a realistic benchmark built from actual classroom data and a principled way to measure whether generated feedback resembles what an instructor would actually write. SEFORA and UniMatch fill both gaps at once, giving researchers a shared yardstick rather than bespoke or self-reported evaluations.
An F1 ceiling of 0.4 is a polite way of saying current models miss more than they hit — and the more they say, the more they miss. That is worth keeping in mind the next time an edtech startup promises AI feedback indistinguishable from a professor's.