Extension to block or minibatch Kaczmarz with smoothed momentum

Extend the analysis of KGSM beyond unit batch size to randomized block Kaczmarz or minibatch variants with geometrically smoothed momentum, and derive convergence-rate results analogous to the single-equation case.

Background

The present analysis focuses on batch size one, aligning KGSM with per-equation updates. Block or minibatch approaches are widely used to reduce variance and can interact with momentum in complex ways.

The authors explicitly ask whether their analysis can be extended to variable block or minibatch sizes, motivating theoretical development to support broader applicability.

References

Our analysis considered the case of block size one (or equivalent minibatch size). Can the analysis be extended to Kaczmarz methods with variable block or minibatch size?

— Randomized Kaczmarz with geometrically smoothed momentum  (2401.09415 - Alderman et al., 2024) in Discussion, Limitations and questions — Block methods