Popularity and Quality in Social News Aggregators: A Study of Reddit and Hacker News

Abstract: In this paper we seek to understand the relationship between the online popularity of an article and its intrinsic quality. Prior experimental work suggests that the relationship between quality and popularity can be very distorted due to factors like social influence bias and inequality in visibility. We conduct a study of popularity on two different social news aggregators, Reddit and Hacker News. We define quality as the relative number of votes an article would have received if each article was shown, in a bias-free way, to an equal number of users. We propose a simple poisson regression method to estimate this quality metric from time-series voting data. We validate our methods on data from Reddit and Hacker News, as well the experimental data from prior work. This method works well even though the collected data is subject to common social media biases. Using these estimates, we find that popularity on Reddit and Hacker News is a stronger reflection of intrinsic quality than expected.