X's hybrid AI-human fact-checking notes get better with human input — but not enough people are showing up to make that matter.
X introduced Collaborative Notes as an extension of its Community Notes system: an LLM drafts a note, then human contributors refine it. Researchers analyzed the full corpus of 19,146 collaborative notes and 211,850 instances of human feedback. They found that suggestions involving factual corrections and added context were most likely to be incorporated into revised drafts, while feedback involving subjective policy calls was largely ignored. Notes that received challenges to their main claim — especially from more active contributors — showed the largest helpfulness gains across versions.
The catch: collaborative notes reach "helpful" status and actually appear on the platform at lower rates than notes written by humans alone or AI alone. The bottleneck is participation, not quality. When human feedback does arrive, it moves the needle; the problem is that it often doesn't. That finding matters for any platform betting that a human-in-the-loop layer will fix AI moderation at scale — the loop only closes if humans bother to engage.
Community Notes is now live on X, Facebook, Instagram, Threads, and TikTok, making it one of the more widely deployed crowd-sourced moderation systems in existence. The researchers frame collaborative notes as complementary rather than competitive — they tend to target posts that attract neither human nor AI-only notes, filling a gap rather than replacing existing coverage. Whether that gap-filling role justifies the engineering overhead of a hybrid pipeline, especially when participation remains thin, is a question the paper raises but does not answer.