A new benchmark called MirrorCode asks AI agents to replicate working software without ever seeing the source code.
Researchers introduced MirrorCode to fill a gap in how AI coding ability gets measured. Existing benchmarks skew toward short, contained tasks, and high-profile demos — like an AI writing a C compiler — are hard to compare because they vary in human guidance and aren't standardized. MirrorCode instead gives agents 25 complete programs spanning Unix utilities, bioinformatics tools, interpreters, cryptography, and compression, then checks whether the AI's reimplementation matches the original's output on end-to-end tests, including tests the agent never saw. The strongest model scored 56% across the benchmark. One standout: an AI rebuilt gotree, a 16,000-line bioinformatics toolkit that the researchers estimate would take a human engineer weeks.
The benchmark matters because it shifts the question from "can AI write code?" to "can AI understand what a program does and reproduce it at scale?" That's a harder, more realistic test of autonomous software engineering — and a 56% pass rate on complex, real-world targets is a result the field can't easily dismiss. It also sets a reproducible baseline that one-off demos never could.
The catch is cost: running a single attempt on a large task consumed $2,600 and 19 days of inference. That's not a benchmark you run on a Tuesday to check a model update — and it quietly underlines how far "AI can do software engineering" is from "AI can do software engineering cheaply enough to matter."