Researchers have a new way to catch bad AI-generated code before it runs: read the shape of the model's attention.
The method, called CodeTD, uses topological data analysis - a technique for finding structural patterns in complex data - to examine the attention maps that code-generating LLMs produce while writing. The idea is that when a model's output drifts from what a prompt actually asked for, that mismatch shows up as a detectable pattern in the topology of its attention, before the code is ever compiled or run. The researchers tested it on 10 code LLMs of up to 34 billion parameters, across five programming languages and four standard benchmarks: HumanEval, MBPP, BigCodeBench, and MultiPL-E. CodeTD outperformed recent baseline methods and, notably, transferred well between benchmarks it wasn't tuned on.
This matters because most hallucination-detection in code assistants still happens after the fact - running the code, checking outputs, hoping the test suite catches the drift. A pre-execution check that works off internal model signals could catch subtly wrong or insecure code earlier in the pipeline, before it reaches a test suite or, worse, a pull request. That's a meaningfully different failure mode than the wrong-fact hallucinations plaguing chatbots, since bad code can carry real security consequences.
Whether this becomes more than a benchmark result depends on cost and integration: attention-map analysis needs access to internals most commercial coding assistants don't expose, and it still has to prove itself against real-world codebases messier than curated eval sets.