Bidirectional vs. Autoregressive Architectures for World Action Models

Determine whether bidirectional architectures or autoregressive architectures are better suited for World Action Models built upon pretrained video diffusion backbones that jointly predict future video frames and robot actions, with respect to modality alignment between video and action, error accumulation in closed-loop control, and inference efficiency.

Background

The paper introduces World Action Models (WAMs) that jointly predict video and actions using a pretrained video diffusion backbone. A central design decision concerns whether the model should be bidirectional (processing fixed-length sequences) or autoregressive (leveraging KV caching and visual history).

The authors highlight trade-offs: bidirectional models may suffer from modality alignment issues and require subsampling that distorts native frame rates, while autoregressive models can preserve alignment and enable efficient inference but may raise concerns about error accumulation. Establishing which architecture is fundamentally better for WAMs remains unresolved and impacts practical deployment and generalization.