A new study finds that the reward function borrowed from math and coding is actually the worst way to train small AI models to search the web.
Researchers trained Qwen3.5-0.8B, a model with well under a billion parameters, to search Wikipedia and answer open-domain questions using reinforcement learning with verifiable rewards (RLVR), the technique behind recent gains in math and code models. They tested three different reward designs, including the exact-match-only scoring used in the earlier Search-R1 method, training each version with Group Relative Policy Optimization across three separate runs. The best reward setup pushed average exact-match accuracy to 0.352 on a seven-benchmark question-answering suite, up from an untrained baseline of 0.092 - a 3.8-fold gain achieved with no larger teacher model involved. The exact-match-only reward finished last in every run, even on the exact-match metric it was directly optimizing for.
That result cuts against a common assumption in AI research: that a reward recipe proven on large models will simply work at smaller scale. Here, the reward shape considered a safe default for math and coding backfired specifically because the model was small, not despite it. For anyone trying to build cheap, fast search agents without a giant model to distill from, that is a concrete warning about which shortcuts do not transfer.
It is a small, narrow study - one model family, one training dataset - but it is a useful check on the assumption that scaling AI training tricks down is as simple as scaling the model down.