Reinforcement learning pushed small language models close to perfect accuracy on simulated Jira and Confluence API tasks — a result that scales-down rather than scales-up the usual AI playbook.
Researchers built five synthetic environments mimicking the Jira REST v3 and Confluence v2 APIs at schema fidelity, then applied Reinforcement Learning with Verifiable Rewards (RLVR) — a technique where the model earns scores based on whether its tool calls are actually correct, not whether they sound plausible. No live API, no human labels, no learned judge. The training signal came entirely from inspecting the tool-call trace. Applied to Qwen3-1.7B and Qwen3.5-4B, the RL-trained policy lifted average reward from a baseline range of 0.35-0.92 up to 0.95-1.00 across four non-degenerate scenarios. The biggest single jump was Confluence page creation, which went from 0.35 to a perfect 1.00.
The gap the paper targets is real and underappreciated: a model trained to predict the next token has no innate reason to fill required fields, sequence API calls correctly, or stop when a task is done rather than hallucinating a tool that does not exist. Enterprise SaaS workflows are exactly the domain where those silent failures compound — a dropped field in a Jira ticket transition does not throw an error, it just does the wrong thing quietly. The finding that a sub-4B model can be tuned to near-perfect reliability on specific endpoints suggests outcome-optimized small models might be a more practical path for narrow enterprise automation than deploying frontier-scale general models.
The authors are candid about the ceiling: hand-crafting verifiable rewards does not scale past a handful of endpoints, and one of their five scenarios was already saturated by the untuned baseline — which is a useful reminder that not every benchmark gap is a gap worth closing.