A new AI tool does not just transcribe what happens in science classrooms. It scores how much genuine reasoning is happening in the conversation.
Researchers built an automated discourse analysis system, called ADAS, that classifies every teacher and student utterance along two axes: what type of talk it is, and what kind of reasoning it reflects, using categories from their earlier CDAT framework. Because some reasoning types are rare in real transcripts, the team rebalanced their annotated dataset, used an LLM to generate synthetic examples for underrepresented categories, and trained a RoBERTa-base model with two classification heads. They also ran a zero-shot GPT-5.4 baseline for comparison, which scored a macro-F1 of 0.467 on utterance type and 0.476 on reasoning component, correctly sorting well under half of the categories, which gave the fine-tuning approach a clear case.
Manually coding classroom transcripts by hand is slow enough that most research and teacher-coaching programs never do it at scale. With this pipeline, the researchers found a genuinely useful pattern buried in the data: when teachers respond to a student answer with a follow-up question rather than a flat correction, students are far more likely to produce inferential reasoning next. That is the kind of finding a school district could actually act on, not just a benchmark number.
Still, the fact that a state-of-the-art model like GPT-5.4 could not crack 50 percent accuracy on either task without fine-tuning is a reminder that off-the-shelf LLMs remain mediocre judges of nuanced classroom talk.