Decentralized Quantile Regression for Feature-Distributed Massive Datasets with Privacy Guarantees

Abstract: In this paper, we introduce a novel decentralized surrogate gradient-based algorithm for quantile regression in a feature-distributed setting, where global features are dispersed across multiple machines within a decentralized network. The proposed algorithm, \texttt{DSG-cqr}, utilizes a convolution-type smoothing approach to address the non-smooth nature of the quantile loss function. \texttt{DSG-cqr} is fully decentralized, conjugate-free, easy to implement, and achieves linear convergence up to statistical precision. To ensure privacy, we adopt the Gaussian mechanism to provide $(\epsilon,\delta)$-differential privacy. To overcome the exact residual calculation problem, we estimate residuals using auxiliary variables and develop a confidence interval construction method based on Wald statistics. Theoretical properties are established, and the practical utility of the methods is also demonstrated through extensive simulations and a real-world data application.