Standard language models read your words but ignore your history.
Researchers have published a framework challenging a foundational assumption in natural language processing: that each document is independent and unordered. In longitudinal settings — think mental health monitoring or behavioral research — that assumption breaks down badly. The paper proposes updating four parts of the NLP pipeline: evaluation splits that account for generalization across people and time, accuracy metrics that separate individual differences from within-person change, inputs that incorporate prior context by default, and model internals tuned to different levels of behavioral history. Testing on a dataset of 17,000 daily diary transcripts from 238 participants, paired with PTSD symptom severity scores, the authors found that traditional document-level evaluation can produce conclusions that are not just different but directly reversed compared to their longitudinal approach.
The stakes here go beyond academic tidiness. Mental health prediction tools built on standard NLP pipelines could be drawing wrong inferences about real patients — not because the models lack data, but because they are fundamentally misreading its structure. Treating a person's day-14 diary entry the same as their day-1 entry ignores trajectory, which is often the whole point in clinical contexts.
This is a narrow but pointed challenge to the way most NLP benchmarks are constructed — and a reminder that optimizing for document-level accuracy metrics can quietly optimize for the wrong thing entirely.