Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

R4DSG: Relative 4D Scene Graph Memory for Object-Centric Question Answering in Long Egocentric Video

Source

arXiv

arxiv.org/abs/2608.11017v1

Read the source

Source

  • Source name: arXiv
  • Source kind: paper
  • Authors / organization: Ke Ma, Yamin Mao, Weiming Li, Shuai Tan, Yijie Zhong, Hao Chen, Haofen Wang, Meng Wang
  • Venue: arXiv: cs.CV
  • Published: 2026-08-11
  • Collected: 2026-08-12
  • URL: http://arxiv.org/abs/2608.11017v1
  • PDF: https://arxiv.org/pdf/2608.11017v1
  • DOI: 10.1145/3767308.3835995
  • Registry group: academic_apis_and_preprints
  • Evidence role: canonical_academic_index
  • Language: en
  • Country / region: UNKNOWN

Plain-Language Brief

The paper introduces R4DSG, a method to create a compact and queryable memory for long egocentric video, which is useful for wearable AI assistants. It focuses on preserving persistent object identity and structured spatial change in object-centric questions. The method separates stable anchors from dynamic objects and represents object state through anchor-relative transitions rather than a globally aligned world model.

What the Document Says

  • The text discusses R4DSG, a method for creating a memory for long egocentric video that is useful for wearable AI assistants.
  • R4DSG focuses on preserving persistent object identity and structured spatial change in object-centric questions.
  • It separates stable anchors from dynamic objects and represents object state through anchor-relative transitions instead of a globally aligned world model.
  • The method is built on recent advances in promptable video segmentation, temporal propagation, and relative 3D lifting.
  • Evaluation shows a significant improvement over existing methods in answering object-related questions.

Key Concepts Mentioned

  • R4DSG
  • long egocentric video
  • wearable AI assistants
  • object-centric questions
  • persistent object identity
  • structured spatial change
  • anchor-relative transitions
  • global world model
  • promptable video segmentation
  • temporal propagation
  • relative 3D lifting

Claims Made by the Document

  • R4DSG introduces a new method for long egocentric video memory that is useful for wearable AI assistants.
  • The method focuses on preserving persistent object identity and structured spatial change in object-centric questions.
  • It produces a retrieval-ready memory directly usable for long-horizon question answering.

Evidence Used

  • The paper describes the R4DSG method and its application for long egocentric video memory.
  • The text mentions the focus on preserving persistent object identity and structured spatial change in object-centric questions.
  • The evaluation results show a significant improvement over existing methods in answering object-related questions.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Long-horizon egocentric video is a rich substrate for wearable AI assistants, but object-centric questions such as where an item was moved, when it last changed state, or why it was relocated remain difficult because caption- and transcript-based memories rarely preserve persistent object identity or structured spatial change. Existing long-video QA methods mainly emphasize temporal grounding and clip retrieval, while prior 3D scene-graph methods typically assume stronger geometry than free-motion wearable RGB video provides, including point clouds, RGB-D input, posed views, sparse reconstruction, or reconstructed scenes. R4DSG introduces a relative 4D scene graph memory for long egocentric video. Instead of storing raw graph sequences, R4DSG converts video into compact queryable memory entries indexed by time, place, persistent objects, anchor-relative change, and local interaction context. The main idea is to separate stable anchors from dynamic objects, maintain persistent object identity across frames, and represent object state through anchor-relative transitions rather than a globally aligned world model. Built on recent RGB-only advances in promptable video segmentation, temporal propagation, and relative 3D lifting, the method produces a retrieval-ready memory directly usable for long-horizon question answering. Evaluation on a 255-question object-related subset from EgoLifeQA shows, under question-only retrieval, a 6.7-point overall gain over EgoRAG-Text and a 12.5-point gain on when questions, which highlights the value of temporally organized object memory. These results position relative 4D scene graphs as a practical memory substrate for wearable assistants, AR systems, and embodied multimedia agents. GitHub Page: https://dualtransparency.github.io/R4DSG/.