Reinforcement learning hits a hard ceiling when AI agents try to execute clinical protocols in real health record environments.
Researchers audited MedAgentBench v1 and v2 — two existing benchmarks for clinical AI — and found that 41.7% of tasks could be silently "finished" without doing anything useful, making inaction the dominant strategy for an RL model optimizing for reward. They built a tighter replacement, MedAgentBench-v3, with 508 tasks and a ceiling of just 8.9%. Training Qwen3-8B on that benchmark exposed two structural problems: a capability ceiling (10 of 20 task types had 0% base performance, giving the model no gradient to learn from) and a format-knowledge barrier (3 task types require exact clinical codes that no amount of exploration will surface). Pure RL reached 18.2% on the benchmark. Rule-based supervised fine-tuning hit 34.1% — a 15.9 percentage-point gap the researchers attribute entirely to those two barriers.
The finding matters because RL from "world feedback" — where a clinical verifier grades rollouts automatically, with no per-episode human annotation — looked like a promising path to scalable medical AI. This paper argues it only works once the base model already knows the codes and formats it needs; RL can learn decision logic, but it cannot bootstrap domain knowledge it was never given. The fix the researchers prescribe is a hybrid: supervised fine-tuning to inject clinical codes, then RL to learn the conditional reasoning on top.
The broader implication is uncomfortable for labs pitching autonomous clinical agents: a benchmark designed to be gamed was being gamed, and fixing it exposed how far current models are from reliable protocol execution.