Peer Truth Serum: Incentives for Crowdsourcing Measurements and Opinions

Abstract: Modern decision making tools are based on statistical analysis of abundant data, which is often collected by querying multiple individuals. We consider data collection through crowdsourcing, where independent and self-interested agents, non-experts, report measurements, such as sensor readings, opinions, such as product reviews, or answers to human intelligence tasks. Since the accuracy of information is positively correlated with the effort invested in obtaining it, self-interested agents tend to report low-quality data. Therefore, there is a need for incentives that cover the cost of effort, while discouraging random reports. We propose a novel incentive mechanism called Peer Truth Serum that encourages truthful and accurate reporting, showing that it is the unique mechanism to satisfy a combination of desirable properties.