Researchers have shown that AI agents designed to teach themselves new skills can just as easily teach themselves bad ones.
A team publishing on arXiv built SARGE, a red-teaming framework that probes self-evolving agents - AI systems that generate, refine, and reuse their own skills from past experience instead of relying on fixed toolsets. Through repeated rounds of generation, escalation, and reinforcement, the researchers coaxed these agents into forming malicious skills and storing them as if they were legitimate capabilities. To make the attack reproducible, they built two benchmarks: EvoSkillBench, a set of malicious interaction trajectories used to induce the bad skills, and EvoSkillSafetyBench, which checks whether those skills get pulled back out and used later. The results were not subtle: injected skills stuck around and were repeatedly activated well after the initial attack.
Skill-based architectures exist to save agents from re-deriving the same reasoning every time, which is exactly why a poisoned skill is so durable - it gets cached, trusted, and reused without a fresh safety check. That turns a one-time jailbreak into a standing capability, closer to planting a backdoor than tricking a chatbot into a bad answer.
Self-evolving agents are still mostly research prototypes, but the direction of travel - agents that build their own toolchains - makes this less a hypothetical and more a design requirement nobody has solved yet.