AI/ ai · interpretability · research · provenance

AI Models Can Leave Traces of Which Internal Path They Used

A new study shows generated text can carry hidden, detectable evidence of which internal computational path a model actually used, not just its final answer.

Text generated by an AI model can secretly carry proof of which internal reasoning path produced it.

Researchers built two small systems, a feed-forward network and a transformer, and forced both through one of two possible internal states before arriving at the same arithmetic answer. They verified which state actually fired, then let that verified state imprint a subtle statistical pattern in the output text. A detector later recovered that pattern with perfect accuracy across all 128 matched test pairs, in both open and separately sealed evaluations, and the effect held up across five independently trained feed-forward models and three independently trained transformers. A follow-up test found that without this deliberate encoding, linear probes could not detect a naturally occurring intermediate state in a plain transformer.

This is an early step toward systems that could prove what a model actually computed internally, not just what it printed, which could eventually help audit AI reasoning or catch deceptive shortcuts. It is a long way from a deployable feature: this is a bounded, engineered proof of concept on a toy arithmetic task, run only on small custom architectures built specifically to carry the signal.

Think of it as a watermark for a chain of thought. As with watermarking research generally, the harder problem is making it survive contact with a real, general-purpose model out in the wild.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →