Researchers spent 39 days inside live classroom polling sessions and found that most polls aren't testing what students know - they're testing whether students were paying attention.
The study tracked 47 live lecture sessions, covering 604 poll questions and 340,668 individual responses from 2,807 students, most of them simple True/False items. Researchers read every poll alongside a time-aligned transcript of the lecture it appeared in, rather than treating each question as a standalone item. That context mattered: in 89% of polls, the correct answer was stated directly somewhere in the lecture, meaning the poll checked recall of something just said rather than independent reasoning. The questions clustered into seven recurring instructional roles, mostly lower-order recall and attention checks, and a separate measure found that when the majority of a class answered a poll, they answered it correctly 88.5% of the time.
Those are two different numbers measuring two different things, and the gap between them is the real finding: a poll can be easy to answer correctly and still be a shallow test, because its difficulty often comes from whether the instructor just said the answer out loud, not from how hard the underlying concept is. A companion survey of 579 students found they could accurately describe what the polls were for and how they participated, but could not judge their own performance accurately - the polls they rated as hardest were not the ones they actually got wrong most often.
Edtech vendors sell live polling as a window into comprehension. This data suggests it is often just a well-disguised attendance check.