Researchers just made mutation testing smarter by letting the AI see the tests it's supposed to be probing.
Mutation testing checks whether a test suite actually works by planting small bugs, or mutants, in code and seeing whether the tests catch them. Traditional rule-based tools like mutmut apply generic transformation rules, which tends to produce mutants that are trivial, redundant, or technically impossible to catch even with a perfect test suite. Researchers tested a different approach: feed a large language model the problem statement, a correct solution, and the existing tests in one prompt, then require it to write a mutant that slips past those tests. Across five models including Gemini 3.1 Pro and GPT 5.1 Codex Mini, tested on the HumanEval and MBPP coding benchmarks, this test-aware prompting produced mutants that were genuine bugs 87.7% and 79.1% of the time, verified against an extended test oracle. The same models prompted without seeing the tests managed only 12.2% and 23.0%, and mutmut trailed at 4.4% and 5.7%.
That gap matters because mutation testing has always had a signal-to-noise problem: teams generate thousands of mutants, most of them useless, and the tool's value gets buried under the noise. Letting a model see what's already covered turns the exercise into something closer to a targeted audit of a test suite's blind spots, and it does so more cheaply per useful mutant than blind prompting, since fewer generated mutants get wasted.
The catch is that HumanEval and MBPP are small, self-contained problems with clean canonical solutions, not the sprawling, half-documented codebases most engineering teams actually test against. The researchers themselves call this a foundation for scaling to production, not proof it already works there.