MyPCBench:一种用于个人智能计算机使用智能代理的测试工具
MyPCBench: A Benchmark for Personally Intelligent Computer-Use Agents
摘要
目前用于评估计算机使用型智能体的标准测试方式都是针对无个人化环境的模型进行的。这就存在一个差距:这些测试无法反映个人助手在实际使用中的表现,因为个人助手需要参与到用户的整个数字生活中,包括各种情境、历史数据以及登录账户等要素。这种差距在网页相关任务中最为明显,因为对于需要登录或涉及个人信息的网站来说,实时网页测试无法真正模拟其实际使用场景。我们开发了MyPCBench这个测试平台,该平台在Linux桌面环境中测试计算机使用型智能体作为个人助手的效果,该环境包含17个模拟的真实网页应用以及完整的桌面工具集,所有测试都基于“办公室”系列剧中的角色迈克尔·斯科特这一典型用户形象。我们在该环境中定义了184项任务,这些任务都源自OpenClaw社区的实际需求;同时,我们还对六种不同类型和规模的模型进行了测试,这些模型都使用了相同的计算机和bash工具环境。结果显示,最好的模型是Claude Opus 4.6,它能够完成55.4%的任务,这是所有模型中唯一超过50%的模型。那些需要处理多个应用程序且任务时间较长的任务,更容易导致模型的失败,因为这些任务对个性化设置的要求更高。我们将在https://mypcbench.com上发布这个测试环境、任务集以及智能体框架。
English Abstract
Current benchmarks for computer-use agents evaluate models in impersonal environments. This leaves a gap between evaluation and deployment where personal assistants are expected to work across a user's whole digital life, including their context, historical data, and logged-in accounts. This gap is widest on web tasks, where live web evaluations cannot exercise sites that require logging in or personal information, the kind of site a real personal assistant has to drive. We introduce MyPCBench, which tests computer-use agents as personal assistants on a Linux desktop populated with 17 simulated real-world web applications and a full desktop stack, all seeded for one canonical persona, Michael Scott from The Office. We define 184 tasks in this environment, each inspired by a real request drawn from the OpenClaw community, and benchmark six closed and open-weight models with a uniform computer+bash tool surface. We find that the best model, Claude Opus 4.6, fully solves 55.4\% of the tasks, the only model above 50\%. Model failures cluster on tasks that span many applications and on long trajectories, where personalization stresses an assistant the most. We release the environment, task set, and agent harness at https://mypcbench.com.