A new benchmark finds that self-evolving AI coding agents mostly can't cope when the physics under their code changes.
PACE-Bench pairs 144 physics tasks across six domains, from statics to dynamics, each with a source environment where a code-based design works and a mutated target environment with the same goal but different physics. Agents have to detect that their design broke, then rewrite it using feedback from a simulator sandbox, all within a capped number of attempts. The researchers tested ten self-evolving methods across four approaches head to head; the best, Reflexion combined with Qwen3-14B, solved just 35.9 percent of the full benchmark, while GPT-5.5 topped out at 66.7 percent on the easiest subset, statics, even with its full attempt budget.
The interesting part isn't just that scores were low. Giving agents the exact physical change that broke their design didn't raise the ceiling either, which points to the real bottleneck: agents can tweak parameters fine, but redesigning a mechanism from scratch is a different skill self-evolution hasn't cracked. That's a pointed rebuttal to the idea that agents which reflect on failures keep getting better at general problem solving.
For a paradigm sold on learning from experience, failing to notice it needs a new blueprint rather than a tweaked one is a rough report card.