Towards Computer-Using Personal Agents

This paper introduces Computer-Using Personal Agents (CUPAs), an extension of current Computer-Using Agents (CUAs). The central proposition is to provide CUAs with controlled access to a user's private data through a modular and user-controlled Personal Knowledge Graph (PKG). The authors argue this approach will enable a new generation of agents capable of complex, personalized, and privacy-preserving automation of online tasks. The vision is motivated through detailed user scenarios, followed by a discussion of the state-of-the-art, the added value, the required agent capabilities, key technical challenges, and a proposed three-stage roadmap for development.

The work presents a very timely and ambitious vision for the future of personal AI agents. The core idea of architecturally separating the agent from a user-controlled Personal Knowledge Graph is powerful and well-aligned with the ongoing efforts. However, in its current form, the manuscript reads more like a position paper or a research manifesto than a full research paper. It remains very broad and high-level, identifying a vast landscape of challenges without providing sufficient technical depth or a clear, feasible path to address them. The vision is compelling, but its feasibility is undermined by a lack of engagement with critical foundational questions. Overall, the paper has the potential to be a significant and impactful contribution, but it needs to be substantially reworked to ground its ambitious vision in a more focused and technically concrete proposal.

Strengths
---------

- The topic is highly relevant and timely, addressing the critical need for user control in an era of increasingly capable AI agents that handle sensitive personal data.

- The proposed architectural separation of the agent (CUA) from the user-controlled data repository (PKG) is a modular and domain-agnostic concept that provides a solid foundation for the overall vision.

- The use of concrete and relatable user scenarios (e.g., planning a dinner, supporting a lifelong learner) is effective in motivating the need for CUPAs and illustrates both their potential benefits and the complex, multi-faceted nature of the problems they aim to solve.

Main Weaknesses and Concerns
----------------------------

[Focus and depth] The paper's primary weakness is its breadth. It touches upon a vast number of concepts and directions but often remains at a high level of abstraction, leaving the impression that the vision is very extensive/horizontal while the challenges dwarf the feasibility. This is particularly event when reaching the middle of the paper, when instead of finding a narrative switch to dig deeper in the proposition, more parentheses are opened instead (opportunities, general challenges, subset of challenges, etc.). To address this, I recommend narrowing down the focus of the paper and keeping the discussion centered around the main strengths and challenges of this vision. One way to do this is to extend and consistently reuse the user scenarios set upfront - not just for motivation but as a continuous thread throughout the paper. By analyzing these scenarios in greater technical detail — for example, by showing a fragment of the PKG for the dinner scenario and tracing how a specific privacy policy would be enforced during negotiation and the main interactions involved — the article could narrow its focus and demonstrate the proposed concepts more concretely while providing a deeper exploration of a subset of the challenges and potential solutions.

[Foundational questions] The paper seems to overlook several critical, practical questions that I believe are central to the CUPA vision. Without addressing these, the proposal may feel unsustainable. Specifically: (1) Data governance: where and by whom will the PKG be stored, managed, exposed to external services? How to promote agency in this picture? (2) Transition: how can this vision be achieved sustainably without requiring a full re-engineering of the current data ecosystem, where data is scattered across legally responsible service providers? and (3) Regulatory landscape: how would the proposed system scale across different legal jurisdictions when it comes to data protection? The authors could strengthen this discussion by considering the potential role of public entities in enabling trusted PKG services, as is being explored in other domains [4].

[Structure and narrative] The paper's structure becomes slightly convoluted and loses focus after the "Added Value" section, with the challenges section feeling general and repetitive. To improve the narrative flow, the authors may consider moving some content in the introduction, while focusing the discussion on the very core challenges and opportunities, thereby linking them directly to the specific capabilities of the CUPA designed to address them. In the current version, challenges are many and their definition should be more contextualized. For example, the point on "Accountability and Liability" reads very general and virtually apply to any system. Some of these points are expanded below, but their discussion still reads superficial.

[On CUPA capabilities]. The "CUPA Capabilities" section lists requirements that require a more nuanced discussion. How were these requirements collected and validated? Are these desiderata generally applicable to any domain / use cases? For example, the assertion that "CUPAs must exhibit a high degree of autonomy" is strong. I would recommend the paper should reframe autonomy not as a mandatory feature but as a configurable capability, allowing users to set the level of independence based on the task or their comfort level. Similarly, the claim that an agent "should always act in the user's interests" may be nuanced. The discussion could also incorporate concepts like computational rationality, which posits that seemingly irrational human behaviours are rational under cognitive bounds, suggesting an aligned agent might better serve the user by emulating this behaviour in some contexts?

Minor Comments and Suggestions
------------------------------

The motivation can be strengthened by more explicitly referencing current, real-world issues of AI misuse of personal data in critical sectors like healthcare [2, 3], which would provide a more urgent context for the proposed solution.

The "Added Value" section (Page 5) has several underdeveloped points. For example, the authors should expand on "Increased granularity" by providing a concrete example of how a CUPA would negotiate data usage beyond a simple "accept all cookies" choice, perhaps showing a mock dialogue or policy exchange.

The paper should discuss its vision for handling derivative knowledge. What is the stance on knowledge inferred by a service provider from mining patterns in a user's data (e.g., predicting shopping habits)? How does the CUPA architecture account for this higher-order knowledge?

The distinction between an individual CUPA operating in isolation and multiple CUPAs negotiating in a group setting should be made more explicit throughout the paper, as these are fundamentally different problems.

The authors should engage more deeply with highly relevant work like Skjæveland et al. [5] (reference [24]), which proposes an ecosystem for Personal Knowledge Graphs. The paper would be much stronger if it explicitly discussed how the CUPA vision builds upon, differs from, or could be implemented within existing architectural proposals like the one presented there.

References
----------

[1] Generative AI and the GDPR. (2024). European Data Protection Supervisor. https://www.edps.europa.eu/system/files/2024-06/24-06-03_genai_orientati...
[2] "Doctors are using unapproved AI software to record patient meetings, investigation reveals". (2024). Sky News. https://news.sky.com/story/doctors-are-using-unapproved-ai-software-to-r...
[3] Ahmad, M., et al. (2024). "Ethical considerations of artificial intelligence in medicine and healthcare". PMC. https://pmc.ncbi.nlm.nih.gov/articles/PMC10960211/
[4] "AI 'doctors' assistant' to speed up appointments a 'gamechanger'". (2024). Gov.uk. https://www.gov.uk/government/news/ai-doctors-assistant-to-speed-up-appo...
[5] Skjæveland, M. G., et al. (2024). "An Ecosystem for Personal Knowledge Graphs: A Survey and Research Roadmap". AI Open. https://www.sciencedirect.com/science/article/pii/S2666651024000044

**Reviewer context**

I enjoyed reading the paper, and my reflections stem from familiarity with the research agenda on Personal Knowledge Graphs, recent Agentic-AI discourse, and the Ethics of Technology, and I find this paper to be a good bridge between the two former topics.

**Summary**

In this paper, Bonatti et al. define a vision for a new evolution of computer-using agents (CUAs) - Computer-Using Personal Agents (CUPAs).
They argue for reliance on semantics-enabled, formalised, and controlled Personal Knowledge Graphs (PKG) as a vehicle for personal data handling.
In such a setting, using PKGs, autonomous CUPAs can contribute to addressing privacy and security concerns related to private information processed by autonomous agents.

They motivate the discussion with two speculative scenarios where CUPAs can be used for daily personal activities.
Further, relying on a survey of current technical and scientific work on CUAs and PKGs, they illustrate their synergies.
They thus enumerate envisioned capacities of such systems and reflect on open challenges from technical (mechanism) and regulatory (privacy and security) sense.

They conclude their work by stating a three-phase roadmap towards the realisation of CUPAs.
With each stage, they intend to increase the capacity of CUPAs, moving from a more individual and closed to a cooperative and interlinked configuration.

**Characteristics +/-**

- **+** (originality) **originality of the main claim** - application of PKGs as a vehicle for collaborative, autonomous agents that assume responsibility over certain human personal affairs promises improved operation on personal data.
- **+** (significance) **quality of identifying the `core challenges`** - authors discuss a useful range of technical aspects of integrating PKGs: i.e., data interoperability, on-the-fly reasoning and negotiation, and ethically: i.e., balance of responsibility, security, privacy, and formulation of behavioral policies.
- **+** (significance) **useful practical role** - specific phase-based roadmap can act as a framework for researchers resonating with and aiming to contribute to the authors' vision.
- **+** (writing quality) **well-written and well structured paper**

- **-** (significance) **depth of reflection on the ethics of CUPAs** - the ethical and socio-technological impacts of introducing CUPAs are not given sufficient recognition.
- **-** (significance) **the overall idea is original, while, imo, some state-of-the-art technical aspects, addressed in other work, are not sufficiently discussed** - e.g., Model Context Protocol (MCP) or Agent2Agent (A2A), protocols used for agent-system and agent-agent interaction have been introduced for CUAs.

**Detailed Feedback**

Given the profound nature of the envisioned systems, I believe that the paper should recognize the discourse on ethical and socio-technical impacts of CUPAs beyond the aspects of personal data, in particular:

- The speculative scenarios (pages 2-3) paint a picture of CUPAs tightly integrated in the human, social fabric. They span a diversity of concerns: human-human interaction, social structures and workplace dynamics, and accessibility of education. However, these aspects are not discussed from an ethical, moral, or organisational perspective, with only a hint at the possible implications with a brief mention of the importance of `value [of introducing the CUPAs] outweighing the potential harms caused` (page 5).
For example: some tasks might be easy to outsource, such as filling in forms, but others might lead to loss of skills, such as those where we understand by doing (situated learning).
- `AI Agents and Agentic Systems: Redefining Human Contribution, Autonomy, Industry Structures, and Governance` (Hughes et al., 2025, 10.1080/08874417.2025.2483832) could be a useful resource, providing an overview of systematic concerns associated with Agentic AI (and CUAs as its sub-form).

From the technical perspective, I believe the paper would benefit from including recent work on CUA-related technology that came out during the review of this paper, for example:

- Page 8 summarises the contemporary CUAs' mode of operation as: `[they] currently rely on existing browser implementations to render an HTML page and then make use of vision models to interact with the page. While that holds for some CUAs, recent work on Model-Context Protocols (MCPs), introduced in November 2024 by Anthropic, has been embraced by CUA scholars, e.g., `LiteCUA: Computer as MCP Server for Computer-Use Agent on AIOS` (Mei et al., 2025, arXiv:2505.18829), branching away from relying on vision models to API integration, which the paper's authors are also advocating for in Phase 2 of the roadmap.
- Phase 3 of the roadmap on Page 8 calls for CUPA-cooperation through `networks of CUPAs interacting in order to complete tasks involving multiple users.` In April 2025, Google Cloud introduced the Agent2Agent (A2A) protocol, addressing the needs of coordinating multi-agent system collaboration. Similarly. In July 2025, Agent Network Protocol (ANP) was also proposed, addressing similar goals.

Extending the paper with a reflection on these trends can help align the roadmap with recent related developments.

Some other potentially relevant references/resources:
- https://ebooks.iospress.nl/doi/10.3233/FAIA240204
- https://www.dtls.nl/fair-data/personal-health-train/
Might be interesting because of the discussions on privacy-preserving interaction of agents with personal data.

**Typographic Notes**

The paper also contains a few minor typographic issues, for example:

- some tools such as OpenAI Operator are mentioned, but not explained
- On Page 2, the line break from step (4.) to (5.) in the `Personalisation in a Lifelong Learning Coach` scenario appears accidental: i.e., `, and` closing the step (4.).
- Figure 2 on Page 3 has spell-checker underlines on words `OnlineNow` (two times) and `microcredential` (once), and the arrow between `Melissa's Lifelong Learning Coach` to `General online sources` appears highlighted, in comparison to other arrows on the illustration.

**Additional Data**

No data supplements the publication.