有趣的主体式机器人学习
Playful Agentic Robot Learning
摘要
当前的智能机器人系统能够编写可执行的“代码即策略”程序，接收反馈并通过多次尝试来调整行为。不过，这些系统仍然主要基于任务驱动：可复用的技能只有在获得明确指令后才会被掌握。我们研究了“有趣智能机器人学习”机制，其中具有身体结构的编码智能体利用自我引导的玩耍过程来持续学习技能，然后再进行后续任务。我们提出了RATs——用于玩耍阶段技能学习的机器人智能体团队。在玩耍过程中，RATs会提出新的、可学习的探索性任务，规划并执行机器人代码策略，验证进展情况，诊断故障，在详细的步骤级反馈下重新尝试，并将成功的执行结果整理成持久的技能库。在测试阶段，智能体可以复用该技能库中的相关技能来应对新任务。在LIBERO-PRO和MolmoSpaces中的实验表明，通过玩耍方式获得的技能能提升后续任务的性能，与没有玩耍或随机玩耍的基线相比，分别提高了20.6和17.0个百分点。此外，这些学到的技能可以通过简单地将它们放入上下文环境中，直接应用到其他“代码即策略”智能体中，从而在不对底层模型进行微调的情况下，使RoboSuite和实际应用场景中的性能分别提升8.9和8.8个百分点。
English Abstract
Current agentic robot systems can write executable Code-as-Policy programs, observe feedback, and revise behavior across multiple attempts, but they remain largely task-driven: reusable skills are acquired only after explicit instructions. We study Playful Agentic Robot Learning, where an embodied coding agent uses self-directed play as a continual skill-learning stage before downstream tasks arrive. We introduce RATs, Robotics Agent Teams designed for play-time skill acquisition. During play, RATs proposes novel yet learnable exploratory tasks, plans and executes robot-code policies, verifies intermediate progress, diagnoses failures, retries with dense, step-level feedback, and distills successful executions into a persistent code skill library. At test time, the agent reuses relevant skills from this frozen library to help solve new tasks. Experiments in LIBERO-PRO and MolmoSpaces show that play-learned skills improve held-out downstream tasks over no-play and random-play baselines, with 20.6 and 17.0 percentage-point gains over CaP-Agent0 on LIBERO-PRO and MolmoSpaces, respectively. Moreover, the learned skills can be plugged into other inference-time Code-as-Policy agents by simply retrieving them into the context, improving RoboSuite and real-world transfer by 8.9 and 8.8 points, respectively, without finetuning the underlying model.