Toxicity and Cultural Entrenchment in Peer-Production Communities: Toward a Handbook on Intelligent System Design

Abstract: Toxicity and abuse are common in online peer-production communities. The social structure of peer-production communities that aim to produce accurate and trustworthy information require some conflict and gate-keeping to spur content production and curation. However, conflict and gate-keeping often devolve into hierarchical power structures which punish newcomers and lock out marginalized groups through entrenched cultural norms. Community administrators often focus on content quality, rather than consideration for all user safety, to promote community growth and survival. Once toxic cultural norms dominate a peer-production community, it is very difficult for community administrators to stop these behaviors from undermining inclusive peer-production. We propose developing a "handbook of intelligent system design" that attempts to frame design protocols to better read user-community culture and accurately distinguish toxic negative interactions from beneficial conflict.