Ensuring veracity of crowd-sourced conference information in Wikidata

Establish effective strategies to ensure the veracity and accuracy of crowd-sourced conference metadata in Wikidata in the presence of potential vandalism and falsified information, while preserving the platform’s openness for community editing.

Background

The paper integrates scholarly conference metadata into Wikidata and highlights both the benefits of community-driven curation and the risk of malicious edits. Although Wikidata maintains provenance for edits and prior work shows that machine learning can detect vandalism, guaranteeing the overall accuracy of crowd-sourced conference information remains unresolved.

This challenge is particularly relevant for the sustainability and reliability of the enriched conference data (e.g., acceptance rates, organizer roles, committee membership) that the authors have populated in Wikidata using LLM-assisted extraction followed by human validation.

References

Ensuring the veracity (i.e., accuracy) of crowd-sourced conference information within Wikidata poses an open challenge and we plan to formulate different strategies as part of our future work.

— Scholarly Wikidata: Population and Exploration of Conference Data in Wikidata using LLMs  (2411.08696 - Mihindukulasooriya et al., 2024) in Conclusions and Future Work, final paragraph