Give an AI agent two unlabeled buttons and it learns which one pays off through trial and error. Give it a button called "jackpot" and a button called "trap," and it stops running the experiment fairly.
A new arXiv paper (2608.16707) introduces what it calls the "semantic bandit," a twist on the classic multi-armed bandit problem used to study exploration versus exploitation. The researchers gave large language model agents textual labels for their possible actions instead of neutral ones, then watched what happened. When a label's implied meaning matched the actual reward, performance improved. When it didn't, performance degraded severely, because the model leaned on the word instead of the evidence. The paper also found a separate quirk: negative rewards pushed models to explore far more than equivalent positive rewards did, which the authors attribute to reward-scale conventions baked into pre-training text.
This matters because LLM agents are increasingly handed real decisions, from routing customer tickets to picking API calls to try, and those decisions almost always come with descriptive labels attached. The paper's finding suggests the label itself becomes an unaccounted-for thumb on the scale, invisible to anyone benchmarking the agent on reward alone. A model that seems to be reasoning well might just be pattern-matching on word choice, and that's the kind of bias that surfaces only after deployment, when the labels stop being friendly.
Classical bandit algorithms don't care what an arm is called; that was the point of formalizing them decades ago. Bolting language onto the interface reopens a door statisticians spent a long time trying to close, and this paper is a useful reminder that fixing it isn't just a prompting problem.