A Symmetric Random Scan Collapsed Gibbs Sampler for Fully Bayesian Variable Selection with Spike-and-Slab Priors

Abstract: We introduce a symmetric random scan Gibbs sampler for scalable Bayesian variable selection that eliminates storage of the full cross-product matrix by computing required quantities on-the-fly. Data-informed proposal weights, constructed from marginal correlations, concentrate sampling effort on promising candidates while a uniform mixing component ensures theoretical validity. We provide explicit guidance for selecting tuning parameters based on the ratio of signal to null correlations, ensuring adequate posterior exploration. The posterior-mean-size selection rule provides an adaptive alternative to the median probability model that automatically calibrates to the effective signal density without requiring an arbitrary threshold. In simulations with one hundred thousand predictors, the method achieves sensitivity of 1.000 and precision above 0.76. Application to a genomic dataset studying riboflavin production in Bacillus subtilis identifies six genes, all validated by previous studies using alternative methods. The underlying model combines a Dirac spike-and-slab prior with Laplace-type shrinkage: the Dirac spike enforces exact sparsity by setting inactive coefficients to precisely zero, while the Laplace-type slab provides adaptive regularization for active coefficients through a local-global scale mixture.