Timeliness, Consensus, and Composition of the Crowd: Community Notes on X

Abstract: This study presents the first large-scale quantitative analysis of the efficiency of X's Community Notes, a crowdsourced moderation system for identifying and contextualising potentially misleading content. Drawing on over 1.8 million notes, we examine three key dimensions of crowdsourced moderation: participation inequality, consensus formation, and timeliness. Despite the system's goal of collective moderation, we find substantial concentration effect, with the top 10% of contributors producing 58% of all notes (Gini Coefficient = 0.68). The observed consensus is rare-only 11.5% of notes reach agreement on publication, while 69% of posts receive conflicting classifications. A majority of noted posts (approximately 68%) are annotated as "Note Not Needed", reflecting the repurposing of the platform for debate rather than moderation. We found that such posts are paradoxically more likely to yield published notes (OR = 3.12). Temporal analyses show that the notes, on average, are published 65.7 hours after the original post, with longer delays significantly reducing the likelihood of consensus. These results portray Community Notes as a stratified, deliberative system dominated by a small contributor elite, marked by persistent dissensus, and constrained by timeliness. We conclude this study by outlining design strategies to promote equity, faster consensus, and epistemic reliability in community-based moderation.