AI agents that solve optimization problems can get better over time by saving skills that worked - but only if they can tell a good skill from a broken one, and nobody hands them an answer key.
A new system called AdmitOR tries to solve that by checking behavioral evidence instead of known answers. It runs candidate solutions from three different model families, prompting strategies, and solver stacks against resampled versions of a problem, then looks for agreement across those runs before deciding whether to accept, abstain, or escalate a skill for human review. Tested against logs from an existing skill-learning system, AdmitOR hit 92.7% admission precision, versus 87.1% for simple majority vote and 72.6% for just checking whether code runs without erroring. Do the error-rate math and that is roughly 1.8 times fewer poisoned admissions than majority vote and about 3.8 times fewer than execution-only checks - a real improvement, though smaller than the paper's own headline multiplier implies. The resulting skill library was also the smallest of the bunch, yet scored highest on macro accuracy across five public benchmarks: 58.4, beating majority vote's 54.8 and even a library built from ground-truth labels at 53.9.
That last number is the interesting part. A filter with no access to correct answers outperformed one that had them, at least on this metric. It suggests that quality control on what an AI learns from itself may matter more than how the training data was labeled in the first place.
The researchers also admit their method's core promise - a calibrated false-discovery guarantee - held up on their own test data but broke down on a fresh, unlabeled stream, largely because some benchmark problems do not actually match their own stated answers. Worth remembering next time a benchmark score gets cited as fact.