Variational Inference for Sparse Poisson Regression

Abstract: We have utilized the non-conjugate Variational Bayesian (VB) method for the problem of sparse Poisson regression model. To provide an approximate conjugacy in the model, the likelihood is approximated by a quadratic function, which provides the conjugacy of the approximation component with the Gaussian prior on the regression coefficient. Three sparsity-enforcing priors are used for this problem. The proposed models are compared with each other and two frequentist sparse Poisson methods (LASSO and SCAD) to evaluate the estimation, prediction, and sparsity performance of the proposed methods. Throughout a simulated data example, the accuracy of the VB methods is computed compared to the corresponding benchmark MCMC methods. It can be observed that the proposed VB methods have provided a good approximation to the posterior distribution of the parameters, while the VB methods are much faster than the MCMC ones. Using several benchmark count response data sets, the prediction performance of the proposed methods is evaluated in real-world applications.