有能力但粗心：使用计算机的智能体是否遵循上下文完整性？
Capable but Careless: Do Computer-Use Agents Follow Contextual Integrity?
摘要
计算机使用代理现在可以代表用户在电子邮件、日历和待办事项等个人应用程序中执行操作。这种跨应用程序的访问功能确实很有用，但也会带来一个被忽视的隐私风险：当代理在某个环境中运行时，它可能会获取其他环境中不适当的信息。因此，我们开发了AgentCIBench这一评估工具，能够将这种风险转化为可执行的、具有确定性的评分场景。我们重点关注了代理系统中的三种常见故障模式：视觉上的冲突——即代理获取了与任务目标相邻的、被禁止的内容；任务模糊导致的过度分享——即代理根据不够明确的提示输入了过多的个人信息；以及接收方不匹配——即代理向不适合接收该内容的对象发送内容。我们对15种前沿代理进行了测试，发现其失败率相当高：15种代理中有11种在超过50%的场景下出现数据泄露现象，平均泄露率为67.9%。即使代理在完整环境中执行任务，这些故障仍然会持续存在。我们发布AgentCIBench的目的是鼓励开发更安全的计算机使用代理，并将上下文相关性的测试作为部署前的安全检查措施。
English Abstract
Computer-use agents (CUAs) now act on a user's behalf across personal applications such as email, calendars, and to-do lists. This cross-application access is useful, but it also creates a privacy risk that has been largely overlooked: when an agent works in one context, it can pull in information from another that is inappropriate in that context. Hence, we introduce AgentCIBench, an evaluation harness that turns this risk into executable, deterministically scored scenarios. We target three common failure modes in CUAs: visual co-location, where the agent pulls in prohibited items that sit next to the task target in the UI; task-ambiguity overshare, where the agent dumps dense personal state in response to an under-specified prompt; and recipient misalignment, where the agent sends content to an addressee for whom it is inappropriate. We evaluate 15 frontier agents and find a surprisingly high failure rate: 11 of 15 leak on more than 50% of scenarios, with an average leakage of 67.9%, and the same failures persist when agents act end-to-end in the environment to complete the task. We release AgentCIBench to encourage the development of safer computer-use agents and position contextual disclosure testing as a pre-deployment safety check.