Differential Set Selection via Confidence-Guided Entropy Minimization

Abstract: This paper addresses the challenge of identifying a minimal subset of discrete, independent variables that best predicts a binary class. We propose an efficient iterative method that sequentially selects variables based on which one provides the most statistically significant reduction in conditional entropy, using confidence bounds to account for finite-sample uncertainty. Tests on simulated data demonstrate the method's ability to correctly identify influential variables while minimizing spurious selections, even with small sample sizes, offering a computationally tractable solution to this NP-complete problem.