因果-RCM：一种统一的教师强制与自我强制方法，用于流式视频生成和交互式世界模型中自回归扩散蒸馏
Causal-rCM: A Unified Teacher-Forcing and Self-Forcing Open Recipe for Autoregressive Diffusion Distillation in Streaming Video Generation and Interactive World Models
摘要
基于因果扩散变换的自回归视频扩散技术，已成为实时视频生成和基于动作的交互式世界模型的重要技术途径。在本研究中，我们将先进的扩散蒸馏框架rCM应用于自回归视频扩散场景。rCM的核心思想在于前向与反向扩散过程的互补性——前者通过一致性模型实现，后者则通过分布匹配蒸馏实现。这一理念同样适用于自回归场景：教师强制训练方法提供了离线、前向扩散的因果训练方式，而自我强制训练则属于基于策略的、反向扩散的优化方式。 我们的贡献包括：(1)通过大量实验证明，作为初始化方法的教师强制CM是目前最理想的自我强制DMD的补充方案；(2)我们实现了基于教师强制方法的连续时间CM的首次应用，例如sCM/MeanFlow，这种方法利用我们自定义的Mask FlashAttention-2 JVP内核，其收敛速度比离散时间CM快10倍；(3)我们提出了Causal-rCM这一高效的、可扩展的扩散蒸馏与因果训练算法框架；(4)我们仅使用合成数据进行训练，就能在帧级和块级两种模式下达到最先进的视频生成性能。 值得注意的是，我们所训练的2步因果Wan2.1-1.3B模型，仅经过1或2次采样步骤后，就能获得VBench-T2V得分84.63分。我们还将Causal-rCM应用于Cosmos 3这一先进的多模态世界基础模型上，该模型具备基于动作的生成能力，从而能够构建出交互式的世界模型。
English Abstract
Autoregressive video diffusion with causal diffusion transformers has emerged as a major paradigm for real-time streaming video generation and action-conditioned interactive world models. In this work, we extend rCM, an advanced diffusion distillation framework, to autoregressive video diffusion. The core philosophy of rCM lies in the complementarity between forward and reverse divergences, represented by consistency models (CMs) and distribution matching distillation (DMD), respectively, in diffusion distillation. This philosophy naturally carries over to the autoregressive setting, where teacher-forcing (TF) provides an offline, forward-divergence causal training paradigm, while self-forcing (SF) corresponds to an on-policy, reverse-divergence refinement. Our contributions are: (1) through extensive experiments, we show that teacher-forcing CM is currently the best complement to self-forcing DMD as an initialization strategy (2) we present the first implementation of teacher-forcing-based continuous-time CMs (e.g., sCM/MeanFlow) for autoregressive video diffusion, enabled by our custom-mask FlashAttention-2 JVP kernel, achieving 10times faster convergence compared to discrete-time CMs (dCMs) (3) we introduce Causal-rCM, a leading, unified, and scalable algorithm-infrastructure open recipe for diffusion distillation and causal training (4) we achieve state-of-the-art streaming video generation performance in both frame-wise and chunk-wise settings, using only synthetic data for training. Notably, our distilled 2-step causal Wan2.1-1.3B model achieves a VBench-T2V score of 84.63 with only 1 or 2 sampling steps. We further apply Causal-rCM to Cosmos 3, an advanced omnimodal world foundation model for physical AI with action-conditioned generation capability, enabling an interactive world model.