Multi-Agent Domain Adaptation (MADA)

• MADA is a domain adaptation framework for multi-agent systems that disentangles domain-invariant and domain-specific features to enhance generalization.
• It employs episodic memory, cross-domain transport mappings, and hierarchical contrastive objectives to adapt to changing agent roles and environments.
• Empirical evaluations reveal significant improvements in win-rates, trajectory prediction accuracy, and collaborative performance across various benchmarks.

Multi-Agent Domain Adaptation (MADA) encompasses methodologies for adapting models in multi-agent systems to new environments, agent mixes, or tasks featuring significant distributional shifts. This adaptation challenge is compounded by structured heterogeneity: agents possess distinct roles, domain attributes differ, and interaction dynamics may change fundamentally between training and deployment. State-of-the-art approaches explicitly address these complexities by modeling domain-invariant and domain-specific representations across agents, leveraging episodic memory, cross-domain transport mappings, and hierarchical contrastive or causal objectives. MADA arises in collaborative-competitive agent learning, multi-source trajectory prediction, and privacy-preserving settings; current frameworks achieve generalization both by disentangling factors of variation and by simulating unseen target domains during training.

1. Formal Definition and Scope

In MADA, the primary goal is to train models that generalize across multiple domains containing interacting agents whose policies, environmental contexts, and latent roles may be unknown or heterogeneous at test time. Formally, let $\mathcal{N} = \{1, \dots, N\}$ index agents over a global state space $\mathcal{S}$, observation spaces $\mathcal{O}^i$, and action spaces $\mathcal{A}^i$. Each agent may belong to distinct teams (collaborative or competitive) and each domain may present different substrates (tasks, rules, spatial constraints).

Domain adaptation tasks are partitioned as follows:

• Collaborative/competitive MARL generalization: Adapt a learner $\pi^1$ to maximize expected team utility across different substrate environments (SEs), unknown teammate policies, and unknown opponent strategies (Wang et al., 20 Jun 2025).
• Multi-source trajectory prediction: Predict future agent trajectories in a held-out domain using labeled trajectories from multiple source domains (Qian et al., 2023, Xu et al., 19 Sep 2025).
• Federated multi-source adaptation: Aggregate models trained independently on disjoint labeled domains to improve performance on a target without sharing raw data (Ghannou et al., 2024).

A central theme is the explicit modeling of inter-domain and inter-agent factors that break the i.i.d. assumption, requiring disentangled approaches for robust domain transfer.

2. Modeling Strategies and Feature Disentanglement

Approaches to MADA systematically disentangle latent features into domain-invariant and domain-specific subspaces:

• Structural causal models and feature extractors: In "AdapTraj" (Qian et al., 2023), the causal formulation introduces latent variables for each agent ("focal" and "neighbor") that are either invariant or specific to a domain. Formally, features $H_i^i$, $H_{\mathcal{E}_i}^i$ (invariant), $H_i^s$, $H_{\mathcal{E}_i}^s$ (specific) are extracted by shared and per-domain expert modules; a learned aggregator $\mathcal{A}$ simulates unseen target domain-specific features via masking.
• Role and domain-aware adapters: "AdaSports-Traj" (Xu et al., 19 Sep 2025) modulates agent latent encodings with dedicated embeddings for role $r_i$ and domain $d$, followed by cross-attention and gating; hierarchical contrastive heads separately align representation subspaces to suppress optimization conflicts between role- and domain-level adaptation.
• Retrieval-augmented agent modeling: In the ACCA/MRDG paradigm (Wang et al., 20 Jun 2025), episodic behavioral memory stores agent trajectory encodings; retrieval networks identify similar past agent behaviors to inform adaptation, supported by positional encodings and dynamic parameter generation.

These explicit factorization techniques are essential for avoiding negative transfer, maintaining interpretability, and supporting robust generalization to novel agents and contexts.

3. Algorithmic Frameworks and Training Protocols

MADA solutions adopt staged multi-phase optimization routines tailored to their respective settings:

• Supervised, plug-and-play domain generalization: In AdapTraj (Qian et al., 2023), initial training focuses on backbone and per-domain expert modules, followed by aggregator learning under domain masking and a final fine-tuning phase. Loss components include base trajectory-prediction ($L_{base}$), scale-invariant reconstruction ($L_{recon}$), orthogonality ($L_{diff}$), and domain-adversarial similarity ($L_{similar}$).
• Contrastive disentanglement: AdaSports-Traj (Xu et al., 19 Sep 2025) introduces hierarchical InfoNCE losses for role ($\mathcal{L}_{role}$) and domain ($\mathcal{L}_{domain}$) similarity, balancing them to encourage separation of gradients and disentangled latent subspaces.
• Collaborative, privacy-preserving multi-source aggregation: In CMDA-OT (Ghannou et al., 2024), sources independently adapt using entropic optimal transport with class regularization, then models are aggregated by FedAvg weighted by pseudo-labeled target validation accuracy. Sinkhorn's algorithm and HOT mapping underpin the transport steps.

Adaptation is generally evaluated under zero-shot or held-out domain generalization, with ablation studies validating the utility of each disentanglement or collaborative component.

4. Episodic Memory, Retrieval, and Dynamic Generation

Advanced MADA frameworks augment the agent modeling via episodic trajectory memory and retrieval:

• In MRDG (Wang et al., 20 Jun 2025), each agent maintains an episodic memory $D_r$ storing $l$-step trajectories $(x_{it}, a_{it})$, where $x_{it}$ are viewpoint-aligned latent encodings. The retrieval network $f_{\theta^{retr}}$ locates the most similar past observations and selects the modal action among retrieved samples.
• Positional encoding is used to address variable team sizes, with sinusoidal projection $\omega_k$ ensuring dimensional consistency.
• Adaptation via hypernetwork: Retrieved actions and positional embeddings are input to a lightweight module $f_{\theta^{hy}}$ which generates dynamic top-layer policy parameters $\theta^{hype}$. This mechanism conditions the policy on current agent mix and team composition.
• Viewpoint Alignment (VA) ensures latent compatibility across agents by minimizing a cross-agent feature alignment cost $\mathcal{C}_i(\theta_v^i)$.

This memory-retrieval-dynamic generation pipeline enables agents to generalize interaction strategies with previously unseen teammates or opponents.

5. Empirical Evaluation and Performance Benchmarks

Recent empirical evaluations establish state-of-the-art performance and robustness for MADA mechanisms:

• Generalization under task, teammate, or opponent shift: In SMAC, Overcooked-AI, and Melting Pot evaluations, MRDG demonstrates substantial improvement over baselines in win-rate, collaboration scores, and social dilemma returns, e.g., MRDG achieves $76\pm6$ win-rate vs. $41\pm6$ for RPM and $58\pm3$ for CSP on SMAC 5m_vs_6m (Wang et al., 20 Jun 2025).
• Cross-domain trajectory prediction: AdaSports-Traj attains lower minADE (e.g., Basketball-U: $4.21$ vs. $4.77$ for best prior) and maintains realism and low out-of-bound rates in unified-to-single settings, validating hierarchical contrastive and role/domain separation (Xu et al., 19 Sep 2025).
• Multi-source generalization: AdapTraj achieves lowest ADE/FDE across all held-out targets, e.g., PECNet+AdapTraj $0.911/1.670$, outperforming single-source and causal-motion baselines (Qian et al., 2023).
• Collaborative source adaptation: CMDA-OT yields $73.25\%$ target accuracy on VLSC (vs. $67.05\%$) and $96.5\%$ on Office-Caltech10 (vs. $96.4\%$) (Ghannou et al., 2024).

Statistical significance is confirmed via non-parametric tests, and negative transfer analysis shows marked improvement with increased domain diversity in AdapTraj.

Framework	Domain Setting	Key Metric	Best Result
MRDG (Wang et al., 20 Jun 2025)	SMAC/Overcooked/M.P.	win-rate / collab / social	76±6 (SMAC), 205±18 (OC-AI)
AdaSports-Traj (Xu et al., 19 Sep 2025)	Basketball/Football/Soccer	minADE₂₀, OOB	4.21 (Basketball-U)
AdapTraj (Qian et al., 2023)	ETH/UCY, SDD, SYI	ADE/FDE	0.911/1.670 (PECNet+ATraj)
CMDA-OT (Ghannou et al., 2024)	VLSC, Office-Caltech	Target Accuracy (%)	73.25% (VLSC), 96.5% (OCT)

6. Limitations, Extensions, and Future Research

Prominent limitations of current MADA frameworks include dependence on explicit role/domain labels at training (AdaSports-Traj), requirement for domain expert modules (AdapTraj), and privacy constraints restricting raw data exchange (CMDA-OT). Plausible future directions involve weak/self-supervised learning of agent roles, implicit domain factorization, group-sparsity regularization in optimal transport, and end-to-end joint optimization in federated settings.

A plausible implication is that architectures combining episodic memory, attention-based adapters, and disentangled feature learning are broadly applicable to other multi-agent adaptation tasks, such as autonomous driving across heterogeneous cities or human-robot interaction in diverse industrial environments.

Extensions under consideration include differential privacy, secure model aggregation, dynamic source selection for communication efficiency, and hierarchical contrastive or adversarial learning for unsupervised domain factor estimation.

7. Conceptual Significance and Relation to Broader Multi-Agent Learning

MADA intersects with multi-agent reinforcement learning for zero-shot coordination, Ad Hoc Teamwork, multi-source domain generalization, and federated learning across non-IID data. Central research groups contributing to the field include those behind the MRDG/ACCA, AdaSports-Traj, AdapTraj, and CMDA-OT frameworks. Significance arises from MADA’s ability to manage complex structured shifts in agent policy, role, and domain, thereby enhancing robustness, interpretability, and adaptability in multi-agent systems beyond conventional single-agent domain adaptation paradigms. MADA methodology is foundational for scalable multi-agent AI deployment under realistic cross-domain and adversarial conditions.