A new LLM framework beat the competition on factoid biomedical questions by treating question types as fundamentally different problems.
Researchers built a pipeline for BioASQ 14b Task B that stops applying one prompting strategy to every question and instead routes yes/no, factoid, and list questions to separate inference procedures. Yes/no questions get snippet shuffling and self-reflection to reduce sensitivity to the order evidence appears. Factoid questions get full-snippet input paired with chain-of-thought prompting to pin down specific biomedical entities. List questions go to a multi-agent setup where separate agents handle extraction, candidate generation, verification, and aggregation. The team tested strategies on BioASQ 13b before applying the final framework to the live BioASQ 14b challenge.
The first-place finish on the factoid subtask in Batch 4 is a concrete result in a domain where hallucinated entities or missed synonyms can carry real clinical stakes. The broader point is that lumping all question types into a single prompting pattern is a design choice, not a law of nature — and this work shows the seams where that shortcut costs accuracy.
Most LLM benchmarking still treats question type as an afterthought. If the performance gap here holds at scale, expect routing-based pipelines to become standard plumbing in any serious biomedical AI stack — though a single subtask win on one batch is a long way from a validated clinical tool.