Variable Selection Using Nearest Neighbor Gaussian Processes

Abstract: We introduce a novel Bayesian approach for variable selection using Gaussian process regression, which is crucial for enhancing interpretability and model regularization. Our method employs nearest neighbor Gaussian processes, serving as scalable approximations of classical Gaussian processes. Variable selection is achieved by conditioning the process mean and covariance function on a random set that represents the indices of contributing variables. A priori beliefs regarding this set control the variable selection, while reference priors are assigned to the remaining model parameters, ensuring numerical robustness in the process covariance matrix. We propose a Metropolis-Within-Gibbs algorithm for model inference. Evaluation using simulated data, a computer experiment approximation, and two real-world data sets demonstrate the effectiveness of our approach.