A Fully Adaptive Frank-Wolfe Algorithm for Relatively Smooth Problems and Its Application to Centralized Distributed Optimization

Abstract: We study the Frank-Wolfe algorithm for constrained optimization problems with relatively smooth and relatively strongly convex objectives. Building upon our previous work, we propose a fully adaptive variant of the Frank-Wolfe method that dynamically adjusts the step size based on both the relative smoothness constant and the Triangle Scaling Exponent (TSE) of the Bregman divergence. Our method does not require prior knowledge of the function parameters and guarantees convergence using only local information. We establish a linear convergence rate under relative strong convexity and provide a detailed theoretical analysis of the proposed adaptive step-size rule. Furthermore, we demonstrate how relative smoothness and strong convexity naturally arise in the setting of centralized distributed optimization. Under a variance-type assumption on the gradients, we show that the global objective becomes relatively strongly convex with respect to the Bregman divergence generated by a local function. This structure allows us to apply our adaptive Frank-Wolfe algorithm, leading to provable acceleration due to an improved relative condition number. We support our theoretical findings with numerical experiments, showing that the proposed method outperforms both non-adaptive and partially adaptive variants, especially in distributed settings.