A new benchmark tests whether AI models can justify their Chinese content-moderation calls, not just make them.
VARM-Bench, introduced in a new research paper, evaluates language models on Chinese abusive-speech moderation by requiring a structured rationale alongside every decision. Each rationale must anchor six specific fields: the target of the abuse, the target's type, how explicitly the target is named, the author's stance, the harmfulness label, and a fine-grained abuse category. A deterministic scoring protocol then checks field correctness, target alignment, output validity, and whether the full record agrees with the final label, without using another LLM as a judge. The researchers tested multiple model families under zero-shot prompting, taxonomy guidance, and structured chain-of-thought supervision.
The headline finding is the uncomfortable one: models that score well on the final abuse or not-abuse label can still get the supporting reasoning wrong in large numbers. That matters because moderation systems increasingly get judged on accuracy alone, when the auditability of a decision, why a post was flagged and against whom, is what regulators and users actually need to trust the call.
Most moderation benchmarks still grade the answer, not the work. VARM-Bench is a reminder that a passing grade on the label can hide a broken argument underneath it.