A research framework is trying to fix one of reinforcement learning's quieter problems: models that parrot expert moves instead of actually thinking.
Researchers introduced MENTOR — Mixed-policy Expert Navigation for Token-level Optimization of Reasoning — to address a specific weakness in Reinforcement Learning with Verifiable Rewards (RLVR). RLVR has become a standard tool for improving how large language models reason through problems, but it hits a wall when base models aren't capable enough to explore effectively on their own. The usual fix is to have models imitate expert-generated reasoning paths end to end. MENTOR takes a different approach: expert guidance kicks in only at critical decision points, not across the entire reasoning chain. The code is publicly available.
The distinction matters because full imitation trades diversity for correctness — the model learns to copy the shape of expert reasoning without developing its own. By limiting expert input to pivotal moments, MENTOR aims to let models internalize strategy rather than surface behavior, which the researchers say produces better exploration and stronger overall performance. That kind of diversity is what separates a model that can generalize from one that just pattern-matches.
The idea echoes debates in chess AI about when human-style heuristics help versus hurt — too much guidance and you get a mimic, not a thinker. Whether MENTOR's selective-nudge approach holds up outside controlled benchmarks is the next question.