Combining Smoothing Spline with Conditional Gaussian Graphical Model for Density and Graph Estimation

Abstract: Multivariate density estimation and graphical models play important roles in statistical learning. The estimated density can be used to construct a graphical model that reveals conditional relationships whereas a graphical structure can be used to build models for density estimation. Our goal is to construct a consolidated framework that can perform both density and graph estimation. Denote $\bm{Z}$ as the random vector of interest with density function $f(\bz)$. Splitting $\bm{Z}$ into two parts, $\bm{Z}=(\bm{X}^T,\bm{Y}^T)^T$ and writing $f(\bz)=f(\bx)f(\by|\bx)$ where $f(\bx)$ is the density function of $\bm{X}$ and $f(\by|\bx)$ is the conditional density of $\bm{Y}|\bm{X}=\bx$. We propose a semiparametric framework that models $f(\bx)$ nonparametrically using a smoothing spline ANOVA (SS ANOVA) model and $f(\by|\bx)$ parametrically using a conditional Gaussian graphical model (cGGM). Combining flexibility of the SS ANOVA model with succinctness of the cGGM, this framework allows us to deal with high-dimensional data without assuming a joint Gaussian distribution. We propose a backfitting estimation procedure for the cGGM with a computationally efficient approach for selection of tuning parameters. We also develop a geometric inference approach for edge selection. We establish asymptotic convergence properties for both the parameter and density estimation. The performance of the proposed method is evaluated through extensive simulation studies and two real data applications.