Researchers have built a training framework that grades AI models against rubrics written by human experts, then uses the failures to target retraining.
The system, called APTER, starts from a set of criteria compiled by domain experts, each representing a specific professional skill a model needs to demonstrate. For each question a model answers, APTER pulls the relevant criteria and turns them into a query-specific checklist that can be scored without a reference answer. Those scores double as a diagnostic: when the same criterion keeps scoring low across many examples, the framework flags it as a persistent weakness and triggers targeted supervised fine-tuning during reinforcement learning, instead of retraining on everything at once. The team tested this on mathematical reasoning and medical question answering across three generations of models.
This matters because most reward models grade for fluency and surface correctness without telling you which specific skill a model is missing. Tying evaluation to stable, named criteria instead of ad hoc per-query rubrics means a lab can actually see where a model keeps failing and fix that, rather than guessing. The reported gains were substantial: up to 15.86 points on math averages and 8.04 points on medical averages over base models.
APTER is a training method for closing specific skill gaps, not a public benchmark, and the strongest evidence so far comes from the same team that built it; independent replication on domains beyond math and medicine would settle whether the gains generalize.