Assigning uncertainties to ROC curves

Determine a statistically sound procedure for assigning uncertainties to the efficiencies that define points on Receiver Operating Characteristic (ROC) curves, enabling meaningful comparison of ROC curves, particularly in the context of ROC curves constructed from counting experiments where efficiencies are computed as fractions of events passing a threshold.

Background

Receiver Operating Characteristic (ROC) curves summarize the trade-off between detection probability and false-alarm probability by plotting signal efficiency against background efficiency as a decision threshold varies. Comparing different detection systems via their ROC curves requires uncertainty estimates on the pointwise efficiencies that form the curve.

The paper focuses on ROC curves derived from counting data and argues that prior treatments often consider a different notion of efficiency, leading to different statistical behavior. The author proposes an operational, frequentist uncertainty propagation approach based on treating the counts of passing and failing events as statistically independent, but explicitly notes that the general assignment of uncertainties to ROC curves has not been settled.