Vinci2: Providing Proactive Assistance in Continuous Egocentric Videos

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Gong Sitong, Tianyu Yan, Caixin Kang, Bo Zheng, Xiang Ruan, Huchuan Lu, Kaipeng Zhang, Yoichi Sato, Yifei Huang
• Venue: arXiv: cs.CV
• Published: 2026-07-13
• Collected: 2026-07-14
• URL: http://arxiv.org/abs/2607.11523v1
• PDF: https://arxiv.org/pdf/2607.11523v1
• DOI: UNKNOWN

Plain-Language Brief

Researchers have developed a proactive assistance system called Vinci2 for continuous egocentric videos. The system, which advances on the existing reactive response system Vinci, aims to offer more context-dependent and proactive help rather than just reacting to user queries. They also introduced EgoServe, a large-scale benchmark for evaluating proactive assistance in continuous egocentric video. The new model, EgoMemo, uses memory-augmented agents that maintain three complementary memory representations for determining whether and how to assist users.

What the Document Says

• Researchers have developed a system called Vinci2 for proactive assistance in continuous egocentric videos.
• Vinci2 advances the on-device assistant Vinci from reactive response toward proactivity.
• EgoServe is a large-scale benchmark for proactive assistance in continuous egocentric video.
• EgoServe comprises over 3,000 service instances organized along 4 temporal memory horizons and 10 service categories.
• EgoMemo is a training-free, memory-augmented agent used by Vinci2.
• EgoMemo maintains three complementary memory representations: multi-scale temporal summaries, a semantic knowledge graph, and visual embedding archives.
• At each timestep, EgoMemo performs retrieval-augmented reasoning to determine whether assistance is warranted and produces contextually grounded responses.

Key Concepts Mentioned

• proactive assistance
• continuous egocentric video
• Vinci2
• EgoServe
• EgoMemo
• memory-augmented agent

Claims Made by the Document

• Vinci2 offers proactive assistance in continuous egocentric videos.
• EgoServe is a large-scale benchmark for proactive assistance in continuous egocentric video.

Evidence Used

• The text states that Vinci2 provides proactive assistance and EgoServe is presented as the first large-scale benchmark for this purpose.

Methods / Approach

• Vinci2 uses a memory-augmented agent called EgoMemo which maintains three complementary memory representations and performs retrieval-augmented reasoning.

Limitations Stated by the Authors

Not stated in source

Limitations Noted by the Agent

Agent interpretation — not from the source.

• The model may struggle to provide proactive assistance effectively if the context is complex or rapidly changing.
• The model's performance may depend on the quality and relevance of the memory representations it maintains.

Open Questions

• How does the Vinci2 system handle cases where the user might not want proactive assistance?
• What are the specific service categories included in EgoServe?

Notes for Future Q&A

May include interpretation. Labeled where it does.

• The text suggests that the system considers the user's history, current activity, and whether assistance would be welcome but does not provide details on how this is implemented.

Raw Abstract / Excerpt

When should an intelligent assistant speak up without being asked? Continuous egocentric video offers rich, evolving context that enables a new form of assistance: one that is proactive rather than merely reactive. Yet existing approaches either wait passively for user queries or treat every detected event as requiring a response, without considering the user's history, current activity, or whether assistance would actually be welcome. We reframe proactive assistance as a context-dependent decision problem: the agent must not only perceive what is happening, but reason over accumulated temporal context to determine when and whether to intervene. To this end, we present Vinci2, a proactive egocentric assistance system that advances the on-device assistant Vinci from reactive response toward proactivity. On the evaluation side, we present EgoServe, the first large-scale benchmark for proactive assistance in continuous egocentric video. EgoServe comprises over 3,000 service instances organized along 4 temporal memory horizons, ranging from immediate safety alerts to long-term habit coaching, across 10 service categories. On the modeling side, we propose EgoMemo, a training-free, memory-augmented agent that maintains three complementary memory representations: multi-scale temporal summaries, a semantic knowledge graph, and visual embedding archives. At each timestep, EgoMemo performs retrieval-augmented reasoning to determine whether assistance is warranted and, if so, produces contextually grounded responses. Experiments demonstrate that EgoMemo establishes strong baselines on EgoServe while remaining competitive on existing egocentric benchmarks. Our benchmark and code are publicly available at \href{https://sitonggong.github.io/EgoServe-page/}{Vinci2}.