无需重新训练：在测试阶段即可将视觉语言动作模型应用于新的任务中。
Retrieve, Don't Retrain: Extending Vision Language Action Models to New Tasks at Test Time
摘要
将视觉-语言-动作策略应用于新任务时，通常需要针对具体任务进行远程操作演示，并对每个任务进行微调。这种适应性调整在数据收集和计算资源方面都会带来较高的成本。在本文中，我们证明了这种针对特定任务的适应性调整成本可以通过检索机制来替代。我们的策略是在目标执行体（如人类手部模型）和更简单的执行体（如视频模型）的配对数据上进行一次训练后，再将其固定下来。在部署新任务时，只需将新的执行体数据添加到检索池中即可。固定的策略会在每个控制步骤中依赖检索到的轨迹来进行决策，因此新任务可以通过索引数据来处理，而无需更新参数。只有当面对全新的、未见过的解耦执行体时，才需要再次进行微调。我们证明，与标准视觉-语言-动作策略相比，检索机制能够提升策略的性能，其效果在基于视频生成的世界动作模型中尤为明显。在PushT平台上，我们研究了检索机制如何为跨执行体的通用性提供可重复使用的运动先验，从而应对各种未知的目标角度。而在RoboTwin 2.0平台上，我们的方法在未知任务上的表现优于其他跨执行体基线算法。此外，我们还在实际机器人平台上验证了该方法的有效性。
English Abstract
Extending a vision-language-action (VLA) policy to a new task typically requires task-specific teleoperated demonstrations and per-task fine-tuning, making adaptation costly in both data collection and compute. In this paper, we show that this target-side per-task adaptation cost can be replaced by retrieval. Our retrieval-augmented policy is trained once on paired demonstrations from the target embodiment (query) and a cheaper embodiment (pool, e.g., human-hand video), then frozen. New tasks are added at deployment by appending pool-side demonstrations to a retrieval pool. The frozen policy conditions on retrieved trajectories at every control step, so new tasks are absorbed by indexing data rather than updating parameters. Fine-tuning is needed only to take on a new, unseen embodiment, not for each new task. We show that retrieval improves policies beyond a specific backbone, including standard VLA policies, but its effect is especially pronounced in Cosmos Policy, a video-generation-based world-action model (WAM). In this setting, retrieval supplies coarse task progression, while the WAM's future-image objective provides an additional visual consistency signal that strengthens the retrieval-conditioned actions. On PushT, we study how retrieval provides a reusable high-level motion prior for cross-embodiment generalization to unseen goal angles, while on RoboTwin 2.0 our method outperforms cross-embodiment baselines on unseen tasks, and we additionally demonstrate the method on a real robot.