A new study of an AI chemistry tutor finds that how it responds to a stuck student matters far more than whether it is simply blocked from giving away the answer.
Researchers analyzed 20,462 student turns from 1,260 real tutoring sessions with a guided LLM chemistry tutor, isolating 6,630 moments where a student hit an impasse: a conceptual error, expressed uncertainty, or an explicit request for help. To compare tutoring styles, they simulated three versions of the tutor on 150 of these impasses. A baseline tutor handed over the answer directly 50.7% of the time. A no-direct-answer tutor asked a follow-up question every single time, no exceptions. The guided tutor varied its response depending on context.
That contextual flexibility turns out to matter. In the real sessions, each additional turn a student spent stuck lowered the odds of recovering on the very next turn by 12.7%. Generic follow-up questions got less useful the longer a student stayed stuck, while directly addressing the specific error got more useful over time: repeating a failed scripted question led to recovery 28.1% of the time, versus 39.8% when the tutor instead addressed the mistake.
So the never-just-give-the-answer guardrail, applied rigidly and forever, is its own kind of failure mode. A tutor that keeps lobbing the same Socratic question at a student who is already lost isn't protecting productive struggle - it's just stalling.