FAPO：多步骤LLM流程的完全自主优化机制
FAPO: Fully Autonomous Prompt Optimization of Multi-Step LLM Pipelines
摘要
多步骤LLM处理流程在检索、推理和格式化等环节之间的交互过程中会出现问题，因此仅通过提示词优化可能无法发现流程中的瓶颈。我们提出了FAPO（完全自主的提示词优化）框架，该框架允许Claude Code在标准化的代码库中优化LLM处理流程。FAPO能够评估整个处理流程、检查各个中间步骤、诊断故障、提出合理的改进方案，并通过评分函数来验证这些改进效果。它首先尝试对提示词进行修改，只有当提示词优化无法解决问题时，才会在允许的范围内调整流程结构，以解决结构上的瓶颈问题。在六个基准测试和三种任务模型中，FAPO在18次模型基准比较中有15次优于基线GEPA。在11次模型基准比较中，FAPO以非重叠的平均pm试验标准偏差范围获胜，平均FAPO-GEPA提升幅度为+14.1个百分点。在六个HoVer和IFBench比较中，当以提示词优先的方式进行优化时，FAPO在所有六次比较中均取得胜利，平均提升幅度为+33.8个百分点。此外，FAPO还提升了安全任务的性能：在CTIBench-RCM这种安全CVE到CWE的任务中，仅使用提示词的FAPO使GPT-5的测试准确率提升了+4.0个百分点，Foundation-Sec-8B-Instruct的准确率提升了+7.1个百分点，Foundation-Sec-8B-Reasoning的准确率则提升了+2.0个百分点。这些结果表明，FAPO是一种适用于通用和安全相关任务的先进处理流程优化技术。
English Abstract
Multi-step LLM pipelines fail through interactions among retrieval, reasoning, and formatting steps, so prompt-only optimization can miss bottlenecks in the chain. We present FAPO (Fully Autonomous Prompt Optimization), a framework that lets Claude Code optimize an LLM pipeline inside a standardized codebase. FAPO evaluates a pipeline, inspects intermediate steps, diagnoses failures, proposes scoped changes, and validates variants repeatedly to optimize against a score function. It first tries prompt edits and, only when prompt optimization appears insufficient, changes chain structure within the permitted scope when attribution identifies a structural bottleneck. Across six benchmarks and three task models, FAPO beats the baseline GEPA in 15 of 18 model-benchmark comparisons. In 11 model-benchmark comparisons, FAPO wins with non-overlapping mean pm trial-standard-deviation ranges, and the mean FAPO-GEPA gain is +14.1 pp. In the six HoVer and IFBench comparisons where prompt-first search escalated to structural changes, FAPO wins all six with a mean gain of +33.8 pp. FAPO also improves performance on security tasks: on CTIBench-RCM, a security CVE-to-CWE task, prompt-only FAPO lifts test accuracy by +4.0 pp on GPT-5, +7.1 pp on Foundation-Sec-8B-Instruct, and +2.0 pp on Foundation-Sec-8B-Reasoning. These results position FAPO as a state-of-the-art pipeline optimization technique for both general-purpose and security-focused tasks.