A new agricultural benchmark is making frontier AI models look less capable than their general scores suggest.
Researchers released PlantExpertVQA, a visual question answering dataset built from 45 open-source sources — including the widely used PlantVillage corpus — totaling 765,186 question-answer pairs across 150,841 images. The dataset covers 38 crop species and 89 disease conditions, with questions sorted into three levels of cognitive complexity and nine categories. Each question was shaped by domain experts and generated through a two-stage automated pipeline: template synthesis from image metadata, then multi-stage linguistic refinement.
The results sting a little for the AI hype cycle. Frontier vision-language models, including recent open-source instruction-tuned multimodal large language models, performed poorly across the board. The more useful finding is the flip side: fine-tuning a compact 2-billion-parameter model on a small slice of the dataset produced substantial gains in every question category. That suggests the gap is not about raw model scale — it is about domain-specific training data, which has been scarce for plant science until now.
Most plant-disease AI work has stayed narrow — classification and detection, not the interactive, reasoning-based diagnosis a farmer or agronomist actually needs. PlantExpertVQA is an attempt to close that gap, though a benchmark without a production deployment is still just a measuring stick. The agriculture AI space has seen similar dataset releases before without triggering the tooling investments that would bring them to the field.