摆脱自我确认陷阱：一种用于代理式体验学习的执行-蒸馏-验证模式
Escaping the Self-Confirmation Trap: An Execute-Distill-Verify Paradigm for Agentic Experience Learning
摘要
基于经验的自我进化对于大型语言模型智能体通过开放环境中的互动来提升性能至关重要。然而，现有的经验学习方法大多依赖于单一智能体循环机制：同一智能体负责执行任务、总结结果以及决定记忆内容。这种模式使得智能体容易陷入“自我确认陷阱”：错误但自洽的轨迹被误认为是有效的经验，从而导致在检索和重用过程中出现累积性错误。为了解决这一问题，我们提出了EDV框架——一种执行-提炼-验证的体系结构，以实现可靠的经验学习。在执行阶段，多个不同的智能体并行探索相同的任务空间，以生成多样化的候选轨迹；在提炼阶段，专门的第三方智能体对这些轨迹进行比较分析，从而生成有效的经验，减少以执行者为中心的总结偏差；在验证阶段，执行组通过共识机制对候选经验进行验证，只有获得认可的体验才会被写入共享或私有记忆中。通过将这三个阶段分离开来，EDV将经验学习从孤立的自我反思转变为协作式构建过程，能够在记忆存储之前过滤掉错误和混乱的内容。我们在tau2-bench、Mind2Web和MMTB三个具有挑战性的长周期基准测试中对EDV进行了评估，结果显示EDV始终优于其他优秀基线模型，这证明了可靠的经验构建对于智能体的自我进化至关重要。我们的代码可以在https://github.com/shidingz/EDV上获取。
English Abstract
Experience-driven self-evolution is critical for large language model (LLM) agents to improve through open-world interaction. However, existing experience learning methods mostly rely on single-agent loops, where the same agent executes tasks, summarizes outcomes, and determines memory content. This setup makes agents vulnerable to the Self-Confirmation Trap: wrong-but-self-consistent trajectories are misidentified as successful experience, leading to cumulative errors during retrieval and reuse. To address this issue, we propose EDV, an Execute-Distill-Verify framework for reliable experience learning. In the Execute stage, multiple heterogeneous agents explore the same task space in parallel to generate diverse candidate trajectories. In the Distill stage, a dedicated third-party agent comparatively analyzes these trajectories to produce candidate experiences, reducing executor-centric summarization bias. In the Verify stage, the execution group validates candidates via a consensus mechanism, and only approved experiences are written into shared or private memory. By decoupling the three stages, EDV transforms experience learning from isolated self-reflection into collaborative construction, filtering erroneous and noisy content before memory insertion. We evaluate EDV on three challenging long-horizon benchmarks: tau2-bench, Mind2Web and MMTB. Results show EDV consistently outperforms strong baselines, validating that reliable experience construction is essential for robust agent self-evolution. Our code is available at https://github.com/shidingz/EDV.