Improved small-sample inference for functions of parameters in the k-sample multinomial problem

Abstract: When the target parameter for inference is a real-valued, continuous function of probabilities in the $k$-sample multinomial problem, variance estimation may be challenging. In small samples or when the function is nondifferentiable at the true parameter, methods like the nonparametric bootstrap or delta method may perform poorly. We develop an exact inference method that applies to this general situation. We prove that our proposed exact p-value correctly bounds the type I error rate and the associated confidence intervals provide at least nominal coverage; however, they are generally difficult to implement. Thus, we propose a Monte Carlo implementation to estimate the exact p-value and confidence intervals that we show to be consistent as the number of iterations grows. Our approach is general in that it applies to any real-valued continuous function of multinomial probabilities from an arbitrary number of samples and with different numbers of categories.