A new training method for AI-assisted molecular optimization outperforms two existing approaches on a standard benchmark by teaching the model to know when it has outgrown its training examples.
Researchers published Active Group Relative Policy Optimization (Active-GRPO) to address a persistent tension in scientific AI training. Standard supervised fine-tuning compresses multi-step reasoning into single answers. Reinforcement learning with verifiable rewards fixes that but suffers from sparse feedback — the model rarely gets a clear signal. A middle-ground method called Reference-guided Policy Optimization anchors learning to curated reference solutions, which helps until the references themselves become the bottleneck. Active-GRPO breaks that ceiling with two coupled mechanisms: one that switches between imitating a reference and reinforcing the model's own better candidates, and one that continuously replaces the reference with the best solution the model has found so far.
The benchmark result — average SRxSim rising from 0.0959 for GRPO and 0.1665 for RePO to 0.1773 for Active-GRPO — is modest in absolute terms but statistically significant across three seeds on LogP, MR, and QED, which are standard proxies for drug-likeness. The real claim here is architectural: a self-upgrading reference target is a cleaner solution to the "weak reference" problem than hand-curating better data.
Molecular optimization is one of the more credible use cases for scientific LLMs, where outputs can actually be verified against chemistry constraints — which makes it a useful pressure test for training methods before they move to messier domains.