Eye-tracking models can now learn more about what you understand without borrowing a language model to do it.
Researchers built LEXIC, a lightweight extension to an existing eye-tracking baseline, to see how much a gaze-only model could improve using nothing more than precomputed word-level signals: GPT-2 surprisal, word frequency, and word length. Two injection methods were tested — direct concatenation and a residual approach that measures how a reader's fixations deviate from expected patterns. Tested on the OneStop reading comprehension task with ensemble training across ten folds, both methods produced statistically consistent gains of 1.8 to 2.2 percentage points in AUROC on unseen text. The concatenation variant added another 2.9 points on unseen readers.
Those numbers matter because the starting gap is brutal: text-aware models using pretrained language models score 56 to 63 percent AUROC on the EyeBench benchmark, while gaze-only models sit at chance. LEXIC does not close that gap, but it narrows it using signals cheap enough to precompute — no transformer inference required at prediction time. That matters for privacy-sensitive or low-resource deployments where feeding eye-tracking data through a cloud language model is off the table.
The residual method's weaker generalization to out-of-distribution readers — gains dropped to 1.8 points at p = 0.19, not statistically significant — suggests the prediction head memorizes training readers rather than capturing something universal. It is a real limitation, and the paper says so plainly, which is more than most benchmark-chasing papers manage.