递归装置自我改进
Recursive Harness Self-Improvement
摘要
在模型与工具共同进化的机制下，这些工具不仅仅是推理过程中的辅助结构，更是能够生成数据的组件；它们的执行过程可以影响未来基础模型的性能。这就促使我们采用“工具在循环中学习”的方法：既要优化工具的性能以提升智能体的即时表现，又要确保其产生的执行记录能够用于未来的模型训练。然而，不断更新由提供者构建的辅助结构既成本高昂又需要大量人力投入。因此，我们研究是否以任务特定的方式优化用户构建的工具，可以在保持计算效率高的同时，只需几次更新操作就能提升执行记录的质量。为此，我们提出了递归工具自我优化算法——该算法将工具视为智能体循环过程的提示级描述，并通过对其自身修改历史的成对反馈来逐步改进它。在涉及量化金融、机器人技术和药学等领域的30项合成机器学习研究任务中，几次递归优化操作就足以显著提升低推理成本智能体的性能水平，其性能甚至能超过高推理成本设置下的水平，同时还将推理成本降低了60%左右。我们发现，这些提升主要源于更有效的智能体间信息传递，从而改善了特定任务的上下文管理，而非需要更长的推理过程。最后，我们将这种行为形式化为一个信息论假设，作为递归工具自我优化算法的隐含优化目标，从而表明递归工具自我优化算法是一种适用于模型与工具共同进化框架下的实用连续学习算法。
English Abstract
Under model--harness co-evolution, harnesses are not merely inference-time scaffolds but data-generating components whose execution traces can shape future foundation models. This motivates harness-in-the-loop learning: optimizing harnesses for both immediate agent performance and the quality of traces used for future model training. However, continually updating provider-built scaffolds is costly and labor-intensive. We therefore investigate whether optimizing user-constructed harnesses in a task-specific manner can improve execution-trace quality while remaining computationally lightweight and requiring only a few update iterations. To this end, we introduce Recursive Harness Self-Improvement (RHI), which represents the harness as a prompt-level specification of the agent loop and iteratively refines it using pairwise feedback over its own revision history. Across 30 synthetic machine-learning research tasks spanning quantitative finance, robotics, and pharmacy, a few RHI iterations suffice to substantially raise the performance ceiling of low-reasoning-effort agents, exceeding the corresponding maximum-reasoning-effort setting while reducing inference cost by up to 60%. We show that these gains arise primarily from improved task-specific context management through more effective inter-agent information flow rather than longer reasoning traces. Finally, we formalize this behavior as an information-theoretic hypothesis for RHI's implicit optimization objective, suggesting RHI as a practical algorithm for continual learning within the paradigm of model--harness co-evolution.