An AI advisor makes you look good right up until it's taken away, according to a new study of a simulated clothing business.
Researchers ran two preregistered experiments in which participants managed a simulated clothing factory, some with help from an LLM advisor and some without. Across both studies, AI-supported participants reported more confidence and a better sense of understanding, while putting in less effort. In the first study (N=200), the advisor raised company value but didn't improve prediction accuracy; once it was withdrawn, previously supported participants only outperformed the control group when judged against their own past choices, not against a neutral default strategy. In the second study (N=198), AI-assisted participants went bankrupt less often and scored slightly higher on a knowledge test, but that edge was mostly explained by staying solvent rather than by learning more.
The uncomfortable finding is that AI help inflated confidence faster than it built competence. One detail suggests a fix: participants who edited the AI's recommendations more often, rather than accepting them outright, performed better once the advisor was removed and scored higher on the knowledge tests. Passive acceptance looked good on paper and taught nothing; active disagreement did the opposite.
It's the same trade-off GPS made with navigation skills, just measured with a spreadsheet instead of a steering wheel.