Quantifying Collective Epistemic Health

Determine reliable, operational metrics and evaluation methodologies for quantifying collective epistemic health in order to assess the systemic, long-term impacts of deploying epistemic AI agents on knowledge repositories and human cognition, including effects such as cognitive deskilling and changes in the integrity of shared knowledge bases.

Background

The paper argues that evaluating epistemic AI agents requires moving beyond individual interactions to assess their broader systemic effects on knowledge ecosystems. It proposes using ecosystem simulations, longitudinal studies, and observational data to understand agents’ impact but acknowledges substantial measurement difficulties.

The authors highlight that some effects, such as cognitive deskilling, may manifest only over long time horizons and that rapid technological evolution complicates evaluation. They explicitly identify the absence of agreed-upon metrics for collective epistemic health as a central open problem.