LaWAM：用于高效动态感知机器人策略的潜在世界动作模型
LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies
摘要
视觉-语言-动作模型（Vision-Language-Action Models, VLAs）利用大规模的视觉-语言预训练来实现语义级机器人控制，但往往无法准确预测机器人动作对场景的影响。而世界-动作模型（World-Action Models, WAMs）则通过以预测的未来状态作为控制依据来弥补这一缺陷。不过，现有的方法通常依赖于计算成本较高的视频生成技术，且需要大量的像素级冗余信息。我们提出了LaWAM模型——一种潜在世界动作模型。该模型通过将预测出的潜在视觉子目标作为控制依据，而不是使用重建后的未来视频数据，从而让机器人能够感知到未来的动态变化。LaWAM的核心是一个基于潜在动作条件的潜在世界模型。我们通过在预训练的视觉基础模型的潜在空间中训练一个潜在动作模型，再利用其前向解码器来预测未来场景的特征，从而得到LaWAM模型。最后，LaWAM将预测出的潜在视觉子目标作为控制依据，从而实现具有动态感知能力的机器人控制。在LIBERO、RoboTwin以及现实世界操控任务中，LaWAM取得了与最先进模型相当甚至更好的成功率，同时保持了较低的推理延迟。LaWAM每次预测一个动作块所需的时间仅为187毫秒，其时钟延迟水平比像素空间模型低了24倍。
English Abstract
Vision-Language-Action models (VLAs) leverage large-scale vision-language pretraining for semantic robot control, but often lack explicit foresight into how robot actions change the scene. World-Action Models (WAMs) address this limitation by conditioning policies on predicted futures, yet existing approaches typically rely on computationally expensive video generation with substantial pixel-level redundancy. We present LaWAM, a Latent World Action Model that exposes predictive dynamics to robot policies through compact latent visual subgoals instead of reconstructed future video. At the core of LaWAM is a latent-action-conditioned Latent World Model (LaWM). We obtain LaWM by training a latent action model in the latent space of a pretrained vision foundation model and repurposing its forward decoder to predict future observation features for scene evolution. LaWAM then conditions action generation on these predicted latent visual subgoals to enable dynamics-aware robot control. LaWAM achieves state-of-the-art or competitive success rates (SRs) across LIBERO (98.6% SR), RoboTwin (91.22% SR), and real-world manipulation tasks while retaining low-latency inference. LaWAM runs in 187 ms per action-chunk prediction and achieves up to 24x lower wall-clock latency than pixel-space WAMs.