OPD-Evolver：通过基于策略的蒸馏方法，培育出具有全面能力的智能体
OPD-Evolver: Cultivating Holistic Agent Evolver via On-Policy Distillation
摘要
记忆已成为自进化智能体的标准基础架构。不过，存储经验与通过学习如何利用这些经验来进化之间存在着差异。现有的记忆型智能体能够存储路径信息、检索相关经验或积累技能，但它们往往缺乏选择有用经验、基于这些经验采取行动、编写可复用知识以及维护不断增长的知识的综合能力。我们提出了OPD-Evolver这一快速与慢速相结合的进化框架，该框架通过基于策略的自蒸馏机制，帮助智能体实现进化。在快速进化阶段，OPD-Evolver与四级记忆层次结构进行交互，从而读取、利用、记录并维护经验，以实现快速的进化。在慢速进化阶段，经过校准的记忆机制以及特殊的“事后洞察”机制，将这四种能力转化为可部署的策略。在多个领域的测试中，OPD-Evolver的表现优于诸如ReasoningBank这样的记忆系统，其性能提升幅度高达11.5%；而基于训练的方法如Skill0则提升了约5.8%。进一步分析表明，OPD-Evolver能够内化高价值的经验和记忆管理机制，使得OPD-Evolver-9B能够在面对像Qwen3.5-397B-A17B和Step-3.5-Flash这样的强大对手时也能表现出色。这表明，OPD-Evolver不仅是一种增强记忆能力的智能体，更是一种真正具有竞争力的智能体进化工具。
English Abstract
Memory has become a standard substrate for self-evolving agents, yet retaining experience is not the same as learning how to evolve through it. Existing memory agents can store trajectories, retrieve reflections, or accumulate skills, but often lack the holistic competence to select useful experience, act on it, write reusable knowledge, and maintain a growing repository. We introduce OPD-Evolver, a slow-fast co-evolution framework that cultivates such an agent evolver through on-policy self-distillation. In the fast loop, OPD-Evolver interacts with a four-level memory hierarchy to read, use, write, and maintain experience for rapid test-time evolution. In the slow loop, outcome-calibrated memory attribution and privileged hindsight distill these four abilities into the deployable policy. Across multi-domain benchmarks, OPD-Evolver surpasses memory systems such as ReasoningBank by up to 11.5%, and training-based methods such as Skill0 by ~5.8%. Further analysis shows that OPD-Evolver internalizes high-value experience and memory management, enabling OPD-Evolver-9B to challenge giant counterparts such as Qwen3.5-397B-A17B and Step-3.5-Flash, pointing beyond memory-augmented agents toward genuinely qualified agent evolvers.