A new research framework called SkillCommit teaches AI agents to build their own skill libraries by testing whether new tricks actually work the same way across cases, not just whether they sound similar.
Researchers describe SkillCommit as an online skill evolution framework that turns an LLM agent's experience into reusable skills without retraining the model itself. Instead of merging new experiences into existing skills based on semantic similarity or an LLM's say-so, SkillCommit first saves each new experience as its own narrow patch. Only when enough patches share a common behavioral mechanism, verified by replaying them across different cases and running an LLM-based check, does the system abstract them into a higher-level skill. It commits that merged skill only if it preserves the validated behavior of everything it absorbed. Tested on three benchmarks - RuleArena, OpenExempt and KOR-Bench - the approach produced consistent performance gains, and the resulting skills transferred across different model sizes and model families.
Most prior attempts at letting agents learn from experience lean on embeddings or an LLM's judgment call to decide what counts as similar enough to merge, a shortcut that can quietly fuse two strategies that look alike but behave differently and tank performance. SkillCommit's insistence on behavioral proof before consolidation is a stricter, costlier standard: every merge has to survive a replay test, not just a vibe check. The cross-model transfer result is the more practically interesting bit, since it hints that skill libraries built this way could move between different LLM providers instead of being stuck with whichever model learned them.
It is a lab result on three benchmarks, not a shipped product, so treat "consistently improves performance" the way you would treat any single paper's numbers: promising, not proven at scale.