Security/ ai-agents · ai-security · llm-safety

A Guard Skill Learns to Police Rogue AI Coding Skills

Researchers built SkillSonar, a self-editing guard skill that cuts task-conditioned attack success in Claude Code and OpenClaw agents.

An AI agent's installed skills can turn hostile only after a real task gives the malicious action cover to look legitimate.

Researchers describe Defense-as-Skill, a defense that runs its guard as its own installable skill rather than a fixed runtime patch. Their system, SkillSonar, watches alongside untrusted skills and checks sensitive actions against what the user actually asked for, then allows, replans, or asks for confirmation. To test it, the team built SCOPE-R, a dataset of 206 confirmed attacks across 6 risk families and 43 benign tasks. They refined the guard using Monte-Carlo Tree Search on rollout feedback, letting it evolve directly on disk.

This matters because most skill security today is pre-install scanning, which cannot catch an attack that only becomes exploitable once a specific task and workspace state exist. Framing defense as an editable, inspectable skill also means the guard can be audited and updated the same way as the skills it watches, rather than living as an opaque part of the agent's core.

The numbers are notable: on repeated GLM-5 runs, attack success dropped from 0.482 to 0.104 for known attack types and from 0.606 to 0.115 for unseen ones. The paper also reports the guard holding up against adaptive attackers and transferring across models it wasn't tuned on. Still, this is one dataset and a research prototype, not a shipped product, so the real test is whether it survives contact with skills nobody designed to be caught.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →