Interplay of conditioning/guidance with biased generalization in diffusion models

Determine how common generation mechanisms used in diffusion models, particularly classifier-free guidance, interact with the biased generalization regime identified in denoising diffusion probabilistic models, and quantify the extent to which these mechanisms amplify training-data-dependent biases and facilitate data extraction.

Background

The paper identifies a biased generalization regime in denoising diffusion probabilistic models where the test loss continues to improve while the generative behavior becomes increasingly dependent on the specific training samples. This regime is detected on real images and in a controlled hierarchical data model using both sample-level and score-level diagnostics, and is attributed to the sequential nature of feature learning.

Beyond establishing this phenomenon, the authors highlight that widely used generation-time mechanisms—specifically conditioning and guidance such as classifier-free guidance—may interact with and potentially amplify data-dependent biases. Understanding this interaction is important for privacy and data provenance, as guidance can steer generation toward features overfitted to particular training samples, potentially enabling data extraction.