Asymptotics for conformal inference

Abstract: Conformal inference is a versatile tool for building prediction sets in regression or classification. We study the false coverage proportion (FCP) in a transductive setting with a calibration sample of $n$ points and a test sample of $m$ points. We identify the exact, distribution-free, asymptotic distribution of the FCP when both $n$ and $m$ tend to infinity. This shows in particular that FCP control can be achieved by using the well-known Kolmogorov distribution, and puts forward that the asymptotic variance is decreasing in the ratio $n/m$. We then provide a number of extensions by considering the novelty detection problem, weighted conformal inference and distribution shift between the calibration sample and the test sample. In particular, our asymptotic results allow to accurately quantify the asymptotic behavior of the errors when weighted conformal inference is used.