GUI与CLI：仅基于屏幕操作或依赖技能的计算机使用型智能体中的执行瓶颈
GUI vs. CLI: Execution Bottlenecks in Screen-Only and Skill-Mediated Computer-Use Agents
摘要
使用计算机的智能体可以通过图形界面或程序化命令界面来执行软件任务。但现有的研究往往将交互方式与任务类型、初始状态、验证机制以及允许的操作方式等因素混为一谈。我们提出了一种包含440种桌面任务、涉及18种应用程序和12种工作流类型的测试基准。在这种控制环境中，仅使用图形界面的智能体和通过命令行进行操作的智能体面临着相同的目标、状态和最终状态验证机制，但只能执行与其界面相关的操作。结果显示，使用图形界面的智能体达到了59.1%的完全通过率，而使用命令行的智能体则只有48.2%的通过率；不过，通过验证机制来辅助技能提升后，命令行的通过率上升到了69.3%。这表明，命令行的不足主要源于技能覆盖范围的不足，而非模型本身的能力问题。这些结果表明，图形界面和命令行存在不同的执行瓶颈：图形界面智能体在长时间的工作流程中受到可靠交互方式的限制，而命令行智能体则受到其技能接口的覆盖范围和可扩展性限制。
English Abstract
Computer-use agents can execute software tasks through either graphical interfaces or programmatic command interfaces, but existing evaluations confound interaction modality with differences in tasks, initial states, verifiers, and permitted actions. We introduce a matched execution-layer benchmark of 440 desktop tasks across 18 applications and 12 workflow categories, where screen-only GUI agents and skill-mediated CLI agents receive identical goals, states, and final-state verifiers while being restricted to modality-native actions. In this controlled setting, the strongest GUI agent reaches a 59.1% full pass rate, outperforming the strongest original-skill CLI agent at 48.2%; however, verifier-guided skill augmentation raises CLI success to 69.3%, showing that much of the CLI deficit comes from incomplete skill coverage rather than model capability alone. These results suggest that GUI and CLI expose different execution bottlenecks: GUI agents are limited by reliable grounded interaction over long-horizon workflows, whereas CLI agents are limited by the coverage and scalability of their skill interfaces.