A personal blog post published Wednesday puts a number on something AI coding tool skeptics have long suspected: feeling faster and being faster are not the same thing.
Writing at intrepidkarthi.com, the author documents an experiment in which developers using AI assistants reported feeling around 20% more productive. Measured output moved the other way — dropping roughly 19%. The post, titled "The gauge broke," argues that AI tools distort the internal feedback signal developers use to assess their own velocity. The self-reported numbers and the measured numbers didn't just diverge; they went in opposite directions.
A near-40-percentage-point swing between perceived and actual productivity is not noise. Teams setting sprint commitments, making staffing decisions, or renewing tool subscriptions based on how fast engineers feel they are moving could be systematically wrong — and wrong in the most dangerous way: confident and unaware. If the perception gap is real, it would also call into question the self-reported surveys that many AI vendors currently treat as proof of value.
This is one blogger's data, not a peer-reviewed study, and the methodology details are sparse. But it joins a growing pile of informal evidence that AI autocomplete creates a sense of flow that doesn't always track with shipped code — and the vendors selling "40% productivity boosts" are not exactly rushing to fund the independent research that would settle it.