Vinci2: Providing Proactive Assistance in Continuous Egocentric Videos

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Gong Sitong, Tianyu Yan, Caixin Kang, Bo Zheng, Xiang Ruan, Huchuan Lu, Kaipeng Zhang, Yoichi Sato, Yifei Huang
• Venue: arXiv: cs.CV
• Published: 2026-07-13
• Collected: 2026-07-14
• URL: http://arxiv.org/abs/2607.11523v1
• PDF: https://arxiv.org/pdf/2607.11523v1
• DOI: UNKNOWN

Plain-Language Brief

The text discusses a new proactive assistance system named Vinci2, which provides assistance in continuous egocentric videos by considering the user's history, current activity, and whether help would be welcome. The authors present EgoServe, a benchmark for evaluating such systems, and EgoMemo, a memory-augmented agent used in Vinci2 for proactive reasoning. Experiments show that EgoMemo performs well on both the new benchmark and existing egocentric benchmarks.

What the Document Says

• Vinci2 is a new proactive assistance system designed for continuous egocentric videos.
• The system considers user's history, current activity, and whether help is welcome before intervening.
• EgoServe is a large-scale benchmark for proactive assistance in continuous egocentric video.
• EgoServe consists of over 3,000 service instances across 10 service categories and 4 temporal memory horizons.
• EgoMemo is a memory-augmented agent used in Vinci2 for proactive reasoning.
• EgoMemo maintains three complementary memory representations: multi-scale temporal summaries, semantic knowledge graph, and visual embedding archives.
• At each timestep, EgoMemo performs retrieval-augmented reasoning to determine if assistance is needed and produces contextually grounded responses.
• Experiments show that EgoMemo establishes strong baselines on EgoServe and remains competitive on existing egocentric benchmarks.

Key Concepts Mentioned

• proactive assistance
• egocentric video
• Vinci2
• EgoServe
• EgoMemo
• memory-augmented agent
• temporal memory horizons

Claims Made by the Document

• Vinci2 advances on-device assistant Vinci from reactive response to proactivity.
• EgoServe is the first large-scale benchmark for proactive assistance in continuous egocentric video.

Evidence Used

• The text states that Vinci2 'reframes proactive assistance as a context-dependent decision problem' and 'presents Vinci2, a proactive egocentric assistance system'.
• The text mentions 'EgoServe, the first large-scale benchmark for proactive assistance in continuous egocentric video.'.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

When should an intelligent assistant speak up without being asked? Continuous egocentric video offers rich, evolving context that enables a new form of assistance: one that is proactive rather than merely reactive. Yet existing approaches either wait passively for user queries or treat every detected event as requiring a response, without considering the user's history, current activity, or whether assistance would actually be welcome. We reframe proactive assistance as a context-dependent decision problem: the agent must not only perceive what is happening, but reason over accumulated temporal context to determine when and whether to intervene. To this end, we present Vinci2, a proactive egocentric assistance system that advances the on-device assistant Vinci from reactive response toward proactivity. On the evaluation side, we present EgoServe, the first large-scale benchmark for proactive assistance in continuous egocentric video. EgoServe comprises over 3,000 service instances organized along 4 temporal memory horizons, ranging from immediate safety alerts to long-term habit coaching, across 10 service categories. On the modeling side, we propose EgoMemo, a training-free, memory-augmented agent that maintains three complementary memory representations: multi-scale temporal summaries, a semantic knowledge graph, and visual embedding archives. At each timestep, EgoMemo performs retrieval-augmented reasoning to determine whether assistance is warranted and, if so, produces contextually grounded responses. Experiments demonstrate that EgoMemo establishes strong baselines on EgoServe while remaining competitive on existing egocentric benchmarks. Our benchmark and code are publicly available at \href{https://sitonggong.github.io/EgoServe-page/}{Vinci2}.