Generative Regression with IQ-BART

Abstract: Implicit Quantile BART (IQ-BART) posits a non-parametric Bayesian model on the conditional quantile function, acting as a model over a conditional model for $Y$ given $X$. One of the key ingredients is augmenting the observed data ${(Y_i,X_i)}_{i=1}^n$ with uniformly sampled values $\tau_i$ for $1\leq i\leq n$ which serve as training data for quantile function estimation. Using the fact that the location parameter $\mu$ in a $\tau$-tilted asymmetric Laplace distribution corresponds to the $\tau^{th}$ quantile, we build a check-loss likelihood targeting $\mu$ as the parameter of interest. We equip the check-loss likelihood parametrized by $\mu=f(X,\tau)$ with a BART prior on $f(\cdot)$, allowing the conditional quantile function to vary both in $X$ and $\tau$. The posterior distribution over $\mu(\tau,X)$ can be then distilled for estimation of the {\em entire quantile function} as well as for assessing uncertainty through the variation of posterior draws. Simulation-based predictive inference is immediately available through inverse transform sampling using the learned quantile function. The sum-of-trees structure over the conditional quantile function enables flexible distribution-free regression with theoretical guarantees. As a byproduct, we investigate posterior mean quantile estimator as an alternative to the routine sample (posterior mode) quantile estimator. We demonstrate the power of IQ-BART on time series forecasting datasets where IQ-BART can capture multimodality in predictive distributions that might be otherwise missed using traditional parametric approaches.