EEG foundation models hit a wall when moved from the lab to the clinic.
Researchers introduced NeuroAdapt-Bench, a systematic benchmark testing how well EEG foundation models handle distribution shifts — the inevitable differences between the data a model trained on and the messier reality of clinical deployment. They ran representative test-time adaptation (TTA) methods, which let a model adjust to new data during inference without revisiting its training set, across multiple pretrained models, tasks, and datasets. The test conditions ranged from mild out-of-distribution shifts to extreme ones, like swapping a scalp EEG headset for an ear-worn device.
The results are a quiet rebuke to the field: standard TTA approaches yielded inconsistent gains and often made things worse. Gradient-based methods — the dominant flavor of TTA borrowed from computer vision and NLP — were especially prone to degrading performance. Optimization-free approaches held up better, suggesting that the adaptation playbook from other domains does not transfer cleanly to neural signals. That matters because EEG is one of the more realistic paths to affordable, scalable brain-computer interfaces and clinical monitoring.
Healthcare AI has a recurring pattern: models perform well in controlled settings and then quietly fall apart when confronted with a different hospital, device, or patient population. EEG compounds that problem because signal quality varies with electrode placement, skull geometry, and even how tired the technician was when applying the cap. NeuroAdapt-Bench at least gives the field a common yardstick — though a benchmark that confirms existing methods are broken is only useful if someone builds better ones.