Capable but Careless: Do Computer-Use Agents Follow Contextual Integrity?

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Anmol Goel, Iryna Gurevych
• Venue: arXiv: cs.AI
• Published: 2026-06-22
• Collected: 2026-06-23
• URL: http://arxiv.org/abs/2606.23189v1
• PDF: https://arxiv.org/pdf/2606.23189v1
• DOI: UNKNOWN

Plain-Language Brief

Researchers present AgentCIBench, a tool for evaluating the privacy risks of computer-use agents (CUAs) that access personal data across various applications. The study finds that most frontier CUAs have a high failure rate in maintaining contextual integrity and leak sensitive information in more than 50% of scenarios.

What the Document Says

• Computer-use agents (CUAs) are now used across personal applications like email, calendars, and to-do lists.
• These agents have a potential privacy risk as they can pull in inappropriate information from other contexts.
• The authors introduce AgentCIBench, a tool for evaluating the privacy risks of CUAs.
• AgentCIBench focuses on three common failures: visual co-location, task-ambiguity overshare, and recipient misalignment.
• 15 frontier agents were evaluated, with 11 leaking on more than 50% of scenarios.
• The average leakage rate was 67.9%, and the same failures persisted in end-to-end tasks.
• AgentCIBench is released to encourage safer CUAs and position contextual disclosure testing as a pre-deployment safety check.

Key Concepts Mentioned

• Computer-use agents
• privacy risk
• contextual integrity
• AgentCIBench
• evaluation harness

Claims Made by the Document

• CUAs have potential privacy risks.
• AgentCIBench can evaluate the privacy risks of CUAs.
• 11 out of 15 frontier agents leak sensitive information in more than 50% of scenarios.
• The average leakage rate is 67.9%.

Evidence Used

• The authors present AgentCIBench as a tool for evaluating CUAs' privacy risks.
• Data on the evaluation of 15 frontier agents is provided, with failure rates and leakage percentages.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Computer-use agents (CUAs) now act on a user's behalf across personal applications such as email, calendars, and to-do lists. This cross-application access is useful, but it also creates a privacy risk that has been largely overlooked: when an agent works in one context, it can pull in information from another that is inappropriate in that context. Hence, we introduce AgentCIBench, an evaluation harness that turns this risk into executable, deterministically scored scenarios. We target three common failure modes in CUAs: visual co-location, where the agent pulls in prohibited items that sit next to the task target in the UI; task-ambiguity overshare, where the agent dumps dense personal state in response to an under-specified prompt; and recipient misalignment, where the agent sends content to an addressee for whom it is inappropriate. We evaluate 15 frontier agents and find a surprisingly high failure rate: 11 of 15 leak on more than 50% of scenarios, with an average leakage of 67.9%, and the same failures persist when agents act end-to-end in the environment to complete the task. We release AgentCIBench to encourage the development of safer computer-use agents and position contextual disclosure testing as a pre-deployment safety check.