Loss Functions for Measuring the Accuracy of Nonnegative Cross-Sectional Predictions

Abstract: Measuring the accuracy of cross-sectional predictions is a subjective problem. Generally, this problem is avoided. In contrast, this paper confronts subjectivity up front by eliciting an impartial decision-maker's preferences. These preferences are embedded into an axiomatically-derived loss function, the simplest version of which is described. The parameters of the loss function can be estimated by linear regression. Specification tests for this function are described. This framework is extended to weighted averages of estimates to find the optimal weightings. Rescalings to account for changes in control data or base year data are considered. A special case occurs when the predictions represent resource allocations: the apportionment literature is used to construct the Webster-Saint Lague Rule, a particular parametrization of the loss function. These loss functions are compared to those existing in the literature. Finally, a bias measure is created that uses signed versions of these loss functions.