Compatibility of Data Augmentation with NOBLE

Characterize which data augmentation strategies, including Mixup and CutMix, are compatible with the NOBLE nonlinear low-rank branch in transformer training, and identify the conditions under which NOBLE’s benefits are preserved or degraded.

Background

The authors find that Mixup/CutMix interferes with NOBLE’s benefits in ImageNet classification, but they have not systematically mapped which augmentations are compatible.

This represents an explicit gap in understanding of how NOBLE interacts with various augmentation techniques, which could guide practitioners in configuring training pipelines.