The GAN is dead; long live the GAN! A Modern GAN Baseline

Abstract: There is a widely-spread claim that GANs are difficult to train, and GAN architectures in the literature are littered with empirical tricks. We provide evidence against this claim and build a modern GAN baseline in a more principled manner. First, we derive a well-behaved regularized relativistic GAN loss that addresses issues of mode dropping and non-convergence that were previously tackled via a bag of ad-hoc tricks. We analyze our loss mathematically and prove that it admits local convergence guarantees, unlike most existing relativistic losses. Second, our new loss allows us to discard all ad-hoc tricks and replace outdated backbones used in common GANs with modern architectures. Using StyleGAN2 as an example, we present a roadmap of simplification and modernization that results in a new minimalist baseline -- R3GAN. Despite being simple, our approach surpasses StyleGAN2 on FFHQ, ImageNet, CIFAR, and Stacked MNIST datasets, and compares favorably against state-of-the-art GANs and diffusion models.

• The paper introduces a regularized relativistic GAN loss that stabilizes training and mitigates mode collapse through local convergence guarantees.
• The paper proposes the R3GAN model, a streamlined baseline that integrates modern architectures with zero-centered gradient penalties for robust performance.
• The paper demonstrates superior results on benchmarks like StyleGAN2 across datasets such as FFHQ and CIFAR-10, advancing practical GAN training methods.

The GAN is Dead; Long Live the GAN: A Modern Baseline GAN

The paper "The GAN is dead; long live the GAN! A Modern Baseline GAN" challenges the prevailing assumptions surrounding Generative Adversarial Networks (GANs). It primarily addresses the conventional belief that GANs are inherently difficult to train due to their susceptibility to issues such as mode collapse and non-convergence. The authors offer a compelling alternative—a streamlined GAN architecture that obviates the need for many incumbent training tricks and stabilizes training through a novel loss function.

At the core of this study is the introduction of a regularized relativistic GAN loss function. This loss function is designed to resolve the problems associated with mode dropping and non-convergence, managing these issues through well-founded mathematical principles rather than a collection of empirical hacks. The derivation of this loss enables local convergence guarantees, which is a significant step forward given the historical difficulties in obtaining such assurances with other relativistic losses in GANs.

The Relativistic Pairing GAN (RpGAN) framework is pivotal in achieving both stability and diversity during training. By incorporating zero-centered gradient penalties (both $R_1$ and $R_2$), the GAN's propensity for stable and convergent training is significantly enhanced. This dual regularization strategy leads to an improved landscape of the GAN's loss function, thereby preventing divergence during the training process.

In demonstrating the efficacy of their approach, the authors develop R3GAN ("Re-GAN"), a modern GAN baseline that forgoes traditional architectural components typical of backbones like StyleGAN2. R3GAN leverages advancements in architecture from modern ConvNets and transformers by integrating features such as resampling layers, grouped convolutions, and network simplifications that draw from the lineage of ResNet architectures. This modernized approach eschews the complex function-specific tricks employed in previous GAN iterations.

Empirically, R3GAN exhibits superior performance to StyleGAN2 on several prominent datasets including FFHQ, ImageNet, CIFAR-10, and Stacked MNIST, and demonstrates competitive results relative to state-of-the-art diffusion models. For instance, it delivers improved Frechet Inception Distance (FID) scores across various image synthesis tasks, attesting to both its qualitative and quantitative advantages.

The implications of this work are multi-faceted. Practically, it paves the way for more robust GAN training by eliminating reliance on fragile optimization tricks, thereby widening the applicability of GANs across diverse domains. Theoretically, it bridges gaps in understanding around the convergence behavior of adversarial losses and showcases the efficacy of principled regularization.

Looking toward future developments, further experiments could explore scaling R3GAN to more complex applications, such as high-resolution image generation or domain-specific data augmentation tasks. Additionally, integrating these principled enhancements with the rich ecosystem of pretrained models and emerging architectures could further elevate the utility and practicability of GANs.

In summary, this paper represents a significant stride toward simplifying and stabilizing GAN training and suggests a promising future trajectory for this class of generative models in the AI landscape.