A fine-tuned RoBERTa-base model reaches competitive scores on standard question answering benchmarks, according to new research posted to arXiv.
The team trained RoBERTa-base on SQuAD 1.1, a well-worn dataset of context-question-answer triples drawn from Wikipedia. After fine-tuning, the model scored 86.84% on ROUGE-L, 28.24% on BLEU, and 95.38% on BERTScore — metrics that measure how closely generated answers match reference text. The core complaint the researchers address is familiar: even when a QA system has the right context in front of it, it still produces vague or off-topic answers. Targeted fine-tuning, they argue, fixes that.
The result is a reminder that careful dataset curation and task-specific training still matter, even as the industry chases ever-larger general-purpose models. SQuAD has been a standard benchmark since 2016, so strong scores there confirm the approach is sound — though they say little about how the model handles noisier, real-world contexts outside Wikipedia prose.
Fine-tuning smaller models on curated data is a pragmatic alternative to deploying massive frontier models, but the tradeoff is narrow generalization — and SQuAD scores alone won't settle that question.