R4DSG: Relative 4D Scene Graph Memory for Object-Centric Question Answering in Long Egocentric Video

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Ke Ma, Yamin Mao, Weiming Li, Shuai Tan, Yijie Zhong, Hao Chen, Haofen Wang, Meng Wang
• Venue: arXiv: cs.CV
• Published: 2026-08-11
• Collected: 2026-08-12
• URL: http://arxiv.org/abs/2608.11017v1
• PDF: https://arxiv.org/pdf/2608.11017v1
• DOI: 10.1145/3767308.3835995
• Registry group: academic_apis_and_preprints
• Evidence role: canonical_academic_index
• Language: en
• Country / region: UNKNOWN

Plain-Language Brief

Researchers present R4DSG, a relative 4D scene graph memory for long egocentric videos that helps answer object-centric questions more effectively. Unlike previous methods, R4DSG preserves persistent object identity and structured spatial changes. It separates stable anchors from dynamic objects and represents object state through anchor-relative transitions, instead of a globally aligned world model.

What the Document Says

• The paper introduces R4DSG, a relative 4D scene graph memory for long egocentric videos.
• R4DSG converts video into compact queryable memory entries indexed by time, place, persistent objects, anchor-relative change, and local interaction context.
• The method produces a retrieval-ready memory directly usable for long-horizon question answering.
• Evaluation on a subset from EgoLifeQA shows improvement in answering object-related questions compared to EgoRAG-Text.

Key Concepts Mentioned

• Relative 4D Scene Graph Memory
• Long Egocentric Video
• Object-Centric Questions
• Persistent Object Identity
• Anchor-Relative Change

Claims Made by the Document

• R4DSG is effective for answering object-centric questions in long egocentric videos.
• R4DSG improves upon EgoRAG-Text for answering certain types of object-related questions.

Evidence Used

• Evaluation on a 255-question object-related subset from EgoLifeQA shows improvement compared to EgoRAG-Text.

Methods / Approach

• Conversion of video into compact queryable memory entries.
• Separation of stable anchors and dynamic objects.
• Representation of object state through anchor-relative transitions.

Limitations Stated by the Authors

The paper does not discuss limitations in detail, but mentions that the method relies on recent advances in promptable video segmentation, temporal propagation, and relative 3D lifting.

Limitations Noted by the Agent

Agent interpretation — not from the source.

The agent's limitations include uncertainty about the generalizability of the results to other types of videos or questions beyond those tested in EgoLifeQA.

Open Questions

• How does R4DSG perform on a wider range of long-horizon video scenarios?
• What are potential improvements for R4DSG's performance in answering why questions?

Notes for Future Q&A

May include interpretation. Labeled where it does.

The paper focuses on the introduction and evaluation of R4DSG for long egocentric video question answering. It does not delve into the details of the underlying video segmentation, temporal propagation, or relative 3D lifting methods.

Raw Abstract / Excerpt

Long-horizon egocentric video is a rich substrate for wearable AI assistants, but object-centric questions such as where an item was moved, when it last changed state, or why it was relocated remain difficult because caption- and transcript-based memories rarely preserve persistent object identity or structured spatial change. Existing long-video QA methods mainly emphasize temporal grounding and clip retrieval, while prior 3D scene-graph methods typically assume stronger geometry than free-motion wearable RGB video provides, including point clouds, RGB-D input, posed views, sparse reconstruction, or reconstructed scenes. R4DSG introduces a relative 4D scene graph memory for long egocentric video. Instead of storing raw graph sequences, R4DSG converts video into compact queryable memory entries indexed by time, place, persistent objects, anchor-relative change, and local interaction context. The main idea is to separate stable anchors from dynamic objects, maintain persistent object identity across frames, and represent object state through anchor-relative transitions rather than a globally aligned world model. Built on recent RGB-only advances in promptable video segmentation, temporal propagation, and relative 3D lifting, the method produces a retrieval-ready memory directly usable for long-horizon question answering. Evaluation on a 255-question object-related subset from EgoLifeQA shows, under question-only retrieval, a 6.7-point overall gain over EgoRAG-Text and a 12.5-point gain on when questions, which highlights the value of temporally organized object memory. These results position relative 4D scene graphs as a practical memory substrate for wearable assistants, AR systems, and embodied multimedia agents. GitHub Page: https://dualtransparency.github.io/R4DSG/.