Randomized Predictive P-values: A Versatile Model Diagnostic Tool with Unified Reference Distribution

Abstract: Examining residuals such as Pearson and deviance residuals, is a standard tool for assessing normal regression. However, for discrete response, these residuals cluster on lines corresponding to distinct response values. Their distributions are far from normality; graphical and quantitative inspection of these residuals provides little information for model diagnosis. Marshall and Spiegelhalter (2003) defined a cross-validatory predictive p-value for identifying outliers. Predictive p-values are uniformly distributed for continuous response but not for discrete response. We propose to use randomized predictive p-values (RPP) for diagnosing models with discrete responses. RPPs can be transformed to "residuals" with normal distribution, called NRPPs by us. NRPPs can be used to diagnose all regression models with scalar response using the same way for diagnosing normal regression. The NRPPs are nearly the same as the randomized quantile residuals (RQR), which are previously proposed by Dunn and Smyth (1996) but remain little known by statisticians. This paper provides an exposition of RQR using the RPP perspective. The contributions of this exposition include: (1) we give a rigorous proof of uniformity of RPP and illustrative examples to explain the uniformity under the true model; (2) we conduct extensive simulation studies to demonstrate the normality of NRPPs under the true model; (3) our simulation studies also show that the NRPP method is a versatile diagnostic tool for detecting many kinds of model inadequacies due to lack of complexity. The effectiveness of NRPP is further demonstrated with a health utilization dataset.