Robust Least Squares Problems with Binary Uncertain Data

Abstract: We propose a Binary Robust Least Squares (BRLS) model that encompasses key robust least squares formulations, such as those involving uncertain binary labels and adversarial noise constrained within a hypercube. We show that the geometric structure of the noise propagation matrix, particularly whether its columns form acute or obtuse angles, implies the supermodularity or submodularity of the inner maximization problem. This structural property enables us to integrate powerful combinatorial optimization tools into a gradient-based minimax algorithmic framework. For the robust linear least squares problem with the supermodularity, we establish the relationship between the minimax points of BRLS and saddle points of its continuous relaxation, and propose a projected gradient algorithm computing $\epsilon$-global minimax points in $O(\epsilon^{-2})$ iterations. For the robust nonlinear least squares problem with supermodularity, we develop a revised framework that finds $\epsilon$-stationary points in the sense of expectation within $O(\epsilon^{-4})$ iterations. For the robust linear least squares problem with the submodularity, we employ a double greedy algorithm as a subsolver, guaranteeing a $(\frac{1}{3}, \epsilon)$-approximate minimax point in $O(\epsilon^{-2})$ iterations. Numerical experiments on health status prediction and phase retrieval demonstrate that BRLS achieves superior robustness against structured noise compared to classical least squares problems and LASSO.