Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

StoryTeller: Training-Free Narrative Grounding for Long-Form Audio Description

Source

arXiv

arxiv.org/abs/2607.11798v1

Read the source

Source

Plain-Language Brief

The authors propose a new framework called StoryTeller for long-form audio description (AD). Unlike modern video-language models, StoryTeller maintains a narrative memory that carries forward story-relevant information across scenes to make descriptions coherent and informative. It doesn't require subtitles, scripts, or specific fine-tuning. The researchers also introduce a new question-answering benchmark called StoryAD-QA to evaluate the effectiveness of their method.

What the Document Says

  • The authors introduce a framework named StoryTeller for long-form audio description.
  • StoryTeller maintains a narrative memory to carry forward story-relevant information across scenes.
  • It doesn't require subtitles, scripts, AD transcripts, aligned captions, character banks, precomputed face identities, or task-specific fine-tuning.
  • StoryTeller can optionally retrieve public movie metadata to resolve names and story context.
  • The method is evaluated using a question-answering benchmark called StoryAD-QA.
  • Experiments show that StoryTeller improves narrative coherence, factual grounding, and story comprehension over strong baselines.

Key Concepts Mentioned

  • long-form audio description
  • narrative memory
  • story-aware
  • StoryTeller
  • StoryAD-QA

Claims Made by the Document

  • StoryTeller improves narrative coherence, factual grounding, and story comprehension over strong baselines.

Evidence Used

  • Experiments on standard AD benchmarks and diverse long-form videos show that StoryTeller consistently improves...

Methods / Approach

  • StoryTeller framework
  • StoryAD-QA question-answering benchmark

Limitations Stated by the Authors

  • UNKNOWN (The text does not provide information about potential limitations as stated by the authors)

Limitations Noted by the Agent

Agent interpretation — not from the source.

  • Possible limitations could be the quality and completeness of public movie metadata used for resolving names and story context.
  • The effectiveness of StoryTeller might depend on the quality and diversity of the long-form videos used in the experiments.

Open Questions

  • How does StoryTeller handle complex scenes with multiple characters and events?
  • What is the impact of different video qualities on StoryTeller's performance?

Notes for Future Q&A

May include interpretation. Labeled where it does.

  • Discuss how StoryTeller handles story complexity and varying video qualities.
  • Explain the method used for semantic filtering and VLM verification.

Raw Abstract / Excerpt

Long-form audio description (AD) requires more than describing visible actions: it must preserve characters, events, relationships, and story context across scenes so that blind and low-vision (BLV) audiences can follow a film. Modern video-language models (VLMs) are effective on short clips, but they often treat each moment independently, producing descriptions that miss who characters are, why events matter, and how the current scene connects to earlier narrative context. We propose StoryTeller, a training-free framework for story-aware long-form AD. Instead of relying only on local visual cues, StoryTeller maintains a verified narrative memory that carries forward story-relevant information across scenes, enabling later descriptions to remain coherent, grounded, and contextually informative. Given only raw video and a movie title, StoryTeller can optionally retrieve public movie metadata to resolve names and story context, while accepting only facts that are supported by the video through semantic filtering and VLM verification. The method requires no subtitles, scripts, AD transcripts, aligned captions, character banks, precomputed face identities, or task-specific fine-tuning. To evaluate whether generated AD preserves narrative information, we introduce StoryAD-QA, a question-answering benchmark that tests whether a language model can answer story-context questions using only the generated descriptions. Experiments on standard AD benchmarks and diverse long-form videos show that StoryTeller consistently improves narrative coherence, factual grounding, and story comprehension over strong baselines in automatic, QA-based, and human evaluations.