A team of researchers built an AI system that writes science test questions from a textbook, then checks its own work before handing them to a teacher.
The system, called TeachMateGPT, targets a well-known flaw in retrieval-augmented generation (RAG) tools: they often generate multiple-choice and short-answer questions from source material even when the material doesn't actually support the answer. TeachMateGPT indexes documents by syllabus chapter and topic rather than chopping them into arbitrary text chunks, then routes each request through a staged pipeline that can refuse to generate a question if it can't find enough supporting evidence. A verification layer called SAVER scores each draft question for faithfulness, relevance, and hallucination risk before a human teacher reviews it. The researchers tested the system on Bangladesh's Class 8 national science textbook (NCTB), producing 198 questions across all 14 chapters, and had three practicing teachers rate the results.
Faithfulness scores jumped from 0.68 to 0.96 compared to a standard RAG baseline, and relevancy climbed from 0.60 to 0.89 - numbers that matter because a hallucinated fact in a graded exam question isn't a minor bug, it's a wrong answer key. The approach also targets a gap most AI-education tools ignore: low-resource, board-exam-structured curricula outside the US and UK textbook market that most RAG research is built around.
It's a narrow, unglamorous fix - question-writing for one national textbook - but it's the kind of unglamorous fix that actually needs solving before anyone trusts an AI-written exam.