InterCMDM: Block-Causal Diffusion for Autoregressive Human Interaction Generation

Abstract: Text-conditioned human interaction generation must capture both long-range temporal causality within each individual and tightly coupled coordination between partners. Existing interaction diffusion models typically denoise full sequences using bidirectional attention, which obscures causality and hinders streaming and long-horizon generation. Autoregressive alternatives enforce causality but often suffer from temporal drift, leading to coordination degradation and unstable interaction dynamics over time. We propose InterCMDM, a block-causal latent diffusion framework for autoregressive two-person interaction generation. InterCMDM introduces a Dual-Stream Causal Diffusion Transformer that maintains separate causal streams for each person while modeling inter-person dependencies via unified dual-stream attention with multi-task attention masks. These masks unify interaction modeling within a single attention mechanism and support diverse coordination behaviors, including simultaneous actions, reactive responses, leader-follower dynamics, and independent motion. By training a single model across these mask configurations as a form of data augmentation, InterCMDM enables controllable interaction generation by simply selecting the desired attention mask at inference time. Finally, a block-wise diffusion objective enables stable latent rollout over long sequences without repeated decode-encode cycles. InterCMDM achieves state-of-the-art performance on InterHuman and Inter-X, improving text-motion alignment, realism, and long-horizon continuity.

• The paper presents a novel block-causal diffusion framework that enforces temporal causality in text-conditioned two-person interaction generation.
• It employs a dual-stream causal transformer with multi-task attention masks to achieve state-of-the-art text–motion alignment and long-horizon stability.
• Experimental evaluations show improved R-Precision, reduced artifacts, and efficient latent rollout over existing autoregressive methods.

InterCMDM: Block-Causal Diffusion for Autoregressive Human Interaction Generation

Introduction

InterCMDM advances the field of text-conditioned human interaction synthesis by integrating structured temporal causality within each individual actor while robustly modeling dynamic inter-person dependencies. The method addresses two primary limitations observable in the literature: diffusion models rely on bidirectional, full-sequence denoising (violating causality and reducing utility for streaming/long-horizon generation), and autoregressive approaches suffer from temporal drift and degraded partner coordination. InterCMDM proposes a block-causal diffusion architecture, enabling controllable, stable, and semantically aligned two-person interaction generation.

Figure 1: Comparison of existing interaction generation methods and InterCMDM. InterCMDM uniquely uses block-wise autoregressive generation with dual causal streams and interaction masks, as opposed to prior full-sequence, non-causal approaches.

Methodology

Block-Causal Latent Diffusion

InterCMDM leverages a temporal motion VAE, encoding each actor’s motion into temporally downsampled latent blocks, which are then processed via a block-wise diffusion process. Each block is only conditioned on its own and the partner’s history, never the future, thus enforcing causal dependency and enabling autoregressive rollout.

Dual-Stream Causal Diffusion Transformer (DS-Causal-DiT)

The DS-Causal-DiT maintains two parallel causal streams—one per actor—while capturing cross-person interactions within a unified masked attention mechanism. Per-block, attention masks regulate allowable information flow, encoding temporal and interaction structure (simultaneous, reactive, leader–follower, and independent behaviors). This unification contrasts with prior schemes that either concatenate or process persons separately with distinct modules, sacrificing either granularity of control or coupling strength.

Figure 2: InterCMDM architectural pipeline: latent motion sampling (left), DS-Causal-DiT with unified dual-stream attention and block-wise conditioning (center), and multi-task block attention masks for structured, controllable coordination (right).

Multi-Task Block Attention Masking

A pivotal innovation is the multi-task block attention masking framework. Multiple coordination patterns are embedded as attention mask configurations, and the model is trained with all variants as data augmentation. At inference, users select a mask to control the emergent coordination regime (e.g., enforcing symmetry, leader–follower dynamics, or disabling cross-stream attention entirely). This enables generalization and fine-grained control in a single trained model, unlike bespoke architectures for routine-specific coordination.

Block-Wise Diffusion Objective and Inference

InterCMDM reformulates the diffusion process at the block level. Independently sampled loudness per block, paired with autoregressive attention, provides stable, high-fidelity sequence rollout. Unlike “decode–encode” cycles required by prior autoregressive diffusions, latent rollout is computationally efficient and avoids compounding reconstruction artifacts.

Experimental Evaluation

Quantitative Results

InterCMDM is empirically benchmarked on the InterHuman and Inter-X datasets. Across all fully generative methods, InterCMDM yields state-of-the-art R-Precision, lowest multi-modal distance (MM Dist), and best diversity on both datasets. Notably, InterCMDM surpasses TIMotion, InterMask, InterGen, and InterActor in text–motion alignment and semantic fidelity, improving InterHuman Top-1 R-Precision from 0.501 to 0.529 and Inter-X from 0.411 to 0.523, while also achieving the best balance of FID and diversity.

Long-Horizon and Streaming Generation

Latent rollout with block-wise diffusion demonstrates superior long-horizon stability and transition smoothness compared to prior methods—boundary discontinuities (B/NB) and foot skating artifacts are significantly reduced. Segment-level FID and R-precision remain favorable even as duration increases, and unlike framewise AR methods, no ground-truth prefixes are required.

Figure 3: Qualitative comparisons of long-horizon motion generation for InterCMDM (continuous and coherent) versus InterMask (notable spatial/temporal artifacts).

Ablation and Analysis

• Dual-Stream Architecture: Explicit parallel causal modeling for each individual significantly improves alignment and realism over shared/single-stream designs.
• Parameter Sharing: Person-specific parameters contribute further gains, especially for asymmetric or role-dependent coordination scenarios.
• Multi-Task Masking: Unified, diverse-mask training enables the model to generalize across simultaneous, reactive, leader–follower, and independent patterns. Mask selection at inference allows for direct, on-the-fly control of interaction mode.
• Block Size Tradeoffs: Moderate block sizes (e.g., $B=3$) optimize tradeoffs between streaming latency and performance.

Theoretical and Practical Implications

The adoption of joint causal attention within block-wise diffusion introduces a paradigm for enforcing causal, autoregressive behavior in generative modeling of structured, interactive sequences. The multi-task attention masking mechanism allows a single model to flexibly capture a wide spectrum of coordination motifs, enabling efficient adaptation with minimal inference-time overhead. InterCMDM demonstrates that latent space structuring and dual-stream modeling are critical for mitigating drift, synchronizing partners, and enabling robust, compositionally controlled text-to-interaction generation.

The theoretical implications extend to any domain involving multi-agent temporal generation where causality and interaction motifs are essential (e.g., robotics, embodied AI, social simulation). Practically, the framework supports real-time streaming and compositional motion generation, of clear relevance for autonomous agents, digital avatars, and interactive entertainment.

Future Directions

• Scalability: Extension beyond two agents to multi-person interactions requires generalization of the dual-stream framework, potentially via graph-based or dynamic masking strategies.
• Physical Plausibility: Integration of data-driven or physics-informed contact/avoidance constraints could further boost realism.
• Fine-Grained Control: Augmenting text conditioning with phase-level or agent-specific prompts presents a pathway for more granular user control.
• Data-Driven Mask Learning: Automated discovery or adaptation of interaction masks based on observed motion dynamics is an open research direction.

Conclusion

InterCMDM offers a rigorously causal, autoregressive, and controllable approach to text-driven two-person interaction synthesis by unifying dual-stream temporal modeling and block-wise latent diffusion within a multi-task attention masking paradigm. The resulting model achieves state-of-the-art generation quality, interaction stability, and coordination flexibility, setting a new standard for realistic and structured human interaction generation (2607.01743).