A new training method teaches AI-powered recommendation engines to reward the reasoning behind a guess, not just whether the guess is right.
The system uses "semantic IDs" - short token sequences that stand in for catalog items - so a generative recommender can predict what a shopper wants next the way a language model predicts the next word. Some of these models first write out a short reasoning trace about a user's interests before generating that final ID. Researchers train them with a popular reinforcement-learning method called group-relative policy optimization, which only rewards an exact match between the predicted ID and the real next purchase. In a catalog with millions of items, that reward almost never fires, and when every attempt in a training batch misses, the model learns nothing at all - even attempts with wildly different reasoning get treated identically.
The fix, described in a new arXiv paper, breaks each reasoning trace into individual hypotheses about what a shopper wants and runs each one as a real search query through a separate, frozen retrieval system. If a hypothesis's query would have surfaced the correct item, that specific hypothesis gets credit, independent of whether the final predicted ID was right. That lets the training signal reach the reasoning process itself, not just its output.
This matters because it targets a structural problem, not a bigger model or more data: sparse-reward training breaks down anywhere a system reasons its way to one correct answer among millions of options, from recommendations to search to agentic tool use. The paper reports consistent gains across three Amazon Reviews datasets versus standard training.
Worth noting: the gains are shown only on Amazon product data, and the "oracle" analysis pointing to the biggest upside assumes a system that already knows which of its guesses was the useful one - real deployments won't get that luxury.