Learning in teams: peer evaluation for fair assessment of individual contributions

Abstract: The "free rider" problem has long plagued pedagogies based on collaborative learning. The most common solution to the free rider problem is peer evaluation. As well other existing methods of peer evaluation include self-evaluation --- and hence are prone to grade inflation or, as we show here, are inaccurate in that they do not fairly reward the most hard working student. Another common concern with existing methods of peer evaluation is that students often do not have the necessary skills to evaluate the work of their peers objectively. In this paper, we introduce a new mechanism for peer evaluation that does not rely on self-evaluation, and yet remains accurate, i.e., if all students are completely truthful in their evaluations, then the output of our mechanism becomes an objective truth. At the same time, our mechanism integrates the instructor's judgment with respect to the credibility of students' evaluations. For example, the instructor gives scores to students for writing credible reviews and, in turn, subsequent students' evaluations are weighted according to these instructor scores.