Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

REDDIT: Correcting Model-Generated Timestamp Drift in ASR without Forgetting via Replay-Based Distribution Editing

Source

arXiv

arxiv.org/abs/2607.05364v1

Read the source

Source

Plain-Language Brief

Researchers propose a solution, called REDDIT, to correct timestamp drift issues in automatic speech recognition (ASR) systems that generate timestamps as part of the transcription. The proposed method uses replay-based distribution editing and doesn't require human transcripts or timestamp annotations. The researchers tested their method on Whisper-tiny and showed improvements in long-gap mean Intersection over Union (mIoU) and out-of-domain Automatic Speech Recognition (AAS), while preserving the model's performance on other tasks.

What the Document Says

  • Modern ASR systems can emit timestamps as part of the transcription.
  • The generated timestamps may drift across long non-speech spans.
  • The researchers created gap and long-gap benchmarks to study timestamp drift across 15 evaluated ASR systems.
  • Naive timestamp correction can degrade non-target ASR behavior, exposing a forgetting problem.
  • REDDIT is a proposed two-stage post-training framework that corrects timestamps without causing catastrophic forgetting.
  • REDDIT uses the model's own replayed decoder context for editing timestamp targets and matches the frozen base distribution on non-timestamp tokens.
  • The researchers constructed correction supervision using VAD-trimmed speech spans, inserted non-speech gaps, and known concatenation offsets.
  • On Whisper-tiny, REDDIT improved long-gap mIoU from 38.7% to 95.0%, reduced mixed-gap out-of-domain AAS from 2752 ms to 223 ms, and preserved CV-en MER at 41.3%.
  • REDDIT used 34.9 hours of targeted correction audio and updated only 1.6% of model parameters.

Key Concepts Mentioned

  • ASR
  • timestamps
  • drift
  • REDDIT
  • replay-based distribution editing
  • correction supervision
  • Whisper-tiny

Claims Made by the Document

  • Timestamp drift can occur in modern ASR systems.
  • Naive timestamp correction can degrade non-target ASR behavior.
  • REDDIT corrects timestamps without causing catastrophic forgetting.

Evidence Used

  • The text mentions that generated timestamps may drift away from the audio, but the transcript remains plausible.
  • The text states that naive timestamp correction improves alignment but degrades non-target ASR behavior, exposing a forgetting problem.
  • The text explains that REDDIT avoids catastrophic forgetting because it edits timestamp targets under the model's own replayed decoder context.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Modern autoregressive ASR systems can emit timestamps as decoded tokens, enabling timestamped transcription without frame-level aligners or inference-time post-processing. We show that these generated timestamps can drift across long non-speech spans: the transcript may remain plausible, but the decoded time axis drifts away from the audio. We study this non-speech-induced timestamp drift with self-built gap and long-gap benchmarks across 15 evaluated timestamp-producing ASR and audio-language systems. Naive timestamp-corrected fine-tuning improves alignment but can severely degrade non-target ASR behavior, exposing a forgetting problem. We propose REDDIT(REplay-based Distribution eDITing), a lightweight two-stage post-training framework that corrects timestamps while avoiding this catastrophic forgetting: it first edits timestamp targets under the model's own replayed decoder context while matching the frozen base distribution on non-timestamp tokens, then applies a short edited-prefix refinement stage. In this framework, we construct correction supervision without human transcripts or human timestamp annotations by combining VAD-trimmed speech spans with inserted non-speech gaps and known concatenation offsets. On Whisper-tiny, 34.9 hours of targeted correction audio used and only 1.6% of model parameters updated, raising long-gap mIoU from 38.7% to 95.0% and reducing mixed-gap out-of-domain AAS from 2752 ms to 223 ms while preserving CV-en MER at 41.3% (versus 524.2% for ordinary SFT decoder tuning).