Prediction Using a Bayesian Heteroscedastic Composite Gaussian Process

Abstract: This research proposes a flexible Bayesian extension of the composite Gaussian process (CGP) model of Ba and Joseph (2012) for predicting (stationary or) non-stationary $y(\mathbf{x})$. The CGP generalizes the regression plus stationary Gaussian process (GP) model by replacing the regression term with a GP. The new model, $Y(\mathbf{x})$, can accommodate large-scale trends estimated by a global GP, local trends estimated by an independent local GP, and a third process to describe heteroscedastic data in which $Var(Y(\mathbf{x}))$ can depend on the inputs. This paper proposes a prior which ensures that the fitted global mean is smoother than the local deviations, and extends the covariance structure of the CGP to allow for differentially-weighted global and local components. A Markov chain Monte Carlo algorithm is proposed to provide posterior estimates of the parameters, including the values of the heteroscedastic variance at the training and test data locations. The posterior distribution is used to make predictions and to quantify the uncertainty of the predictions using prediction intervals. The method is illustrated using both stationary and non-stationary $y(\mathbf{x})$.