Generative Autoregressive Transformers for Model-Agnostic Federated MRI Reconstruction

Abstract: Although learning-based models hold great promise for MRI reconstruction, single-site models built on limited local datasets often suffer from poor generalization. This challenge has spurred interest in collaborative model training on multi-site datasets via federated learning (FL) -- a privacy-preserving framework that aggregates model updates instead of sharing imaging data. Conventional FL aggregates locally trained model weights into a global model, inherently constraining all sites to use a homogeneous model architecture. This rigidity forces sites to compromise on architectures tailored to their compute resources and application-specific needs, making conventional FL unsuitable for model-heterogeneous settings where each site may prefer a distinct architecture. To overcome this limitation, we introduce FedGAT, a novel model-agnostic FL technique based on generative autoregressive transformers. FedGAT decentralizes the training of a global generative prior that learns the distribution of multi-site MR images. For high-fidelity synthesis, we propose a novel site-prompted GAT prior that controllably synthesizes realistic MR images from desired sites via autoregressive prediction across spatial scales. Each site then trains its own reconstruction model -- using an architecture of its choice -- on a hybrid dataset augmenting its local MRI dataset with GAT-generated synthetic MR images emulating datasets from other sites. This hybrid training strategy enables site-specific reconstruction models to generalize more effectively across diverse data distributions while preserving data privacy. Comprehensive experiments on multi-institutional datasets demonstrate that FedGAT enables flexible, model-heterogeneous collaborations and achieves superior within-site and cross-site reconstruction performance compared to state-of-the-art FL baselines.