A new RAG chatbot for cancer registrars doubles how well its answers stick to the actual coding rules, citations and all.
Researchers built CRISS, short for Cancer Registry Intelligent Support System, a retrieval-augmented generation chatbot that answers questions about cancer registry coding and staging standards with citations attached. The team built a knowledge base from national registry standards, broke it into tagged passages, and indexed it as dense embeddings so the model could pull the right guidance before answering. They tested open-weight and proprietary models from the Gemini and GPT families, with and without retrieval, on easy, medium, and hard registry questions, using an LLM-as-a-judge protocol to score the answers. RAG versions beat non-RAG versions across the board, especially on the hardest questions, posting mean grounding scores of 0.62, 0.56, and 0.59 across the three difficulty tiers versus 0.29, 0.26, and 0.29 for models working without retrieval.
Cancer registrars, including Oncology Data Specialists, work against coding and staging standards that change often, and a wrong code can ripple into cancer statistics used for research and public health tracking. Doubling grounding scores is a real gain, but the RAG models still topped out around 0.6, meaning a meaningful share of answers were not fully backed by the cited passages - which is exactly why the study frames CRISS as a training and helpdesk aid, not a replacement for the registrar who signs off on the final abstraction.
That's a more modest pitch than most AI healthcare tools make, and given how often citation-generating chatbots in medicine and law have confidently cited things that don't exist, modest looks like the responsible choice here.