Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note2 min read

Beyond U-Net: A Latent-Representation-Aligned Skip-Free Backbone for Flow-Matching Speech Enhancement

Source

arXiv

arxiv.org/abs/2606.24745v1

Read the source

Source

Plain-Language Brief

The authors propose a new method for speech enhancement using flow-matching, which is more efficient than generative models with iterative sampling processes that limit real-time deployment. Their approach involves a skip-free encoder-decoder backbone guided by Latent Representation Alignment (LRA). This model aligns its bottleneck and decoder representations with clean latent features extracted from a frozen Descript Audio Codec, resulting in improved PESQ and perceptual quality on WSJ0-CHiME3 and VoiceBank-DEMAND datasets.

What the Document Says

  • The text discusses a new method for speech enhancement using flow-matching.
  • The iterative sampling process of generative models limits their real-time deployment.
  • Flow Matching is presented as an efficient alternative to generative models.
  • The new method uses a skip-free encoder-decoder backbone guided by Latent Representation Alignment (LRA).
  • This model aligns its bottleneck and decoder representations with clean latent features extracted from a frozen Descript Audio Codec.
  • The codec-aligned supervision promotes compact clean-speech representations while preserving efficient few-step inference.
  • Experiments on WSJ0-CHiME3 and VoiceBank-DEMAND show improved PESQ and perceptual quality.

Key Concepts Mentioned

  • speech enhancement
  • flow-matching
  • Latent Representation Alignment (LRA)
  • encoder-decoder backbone
  • Descript Audio Codec

Claims Made by the Document

  • Flow Matching offers an efficient alternative to generative models for speech enhancement.
  • The proposed model improves PESQ and perceptual quality in speech enhancement.

Evidence Used

  • Experiments on WSJ0-CHiME3 and VoiceBank-DEMAND show improved PESQ and perceptual quality.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Generative models, particularly diffusion and score-based approaches, have recently achieved strong performance in speech enhancement, but their iterative sampling process limits real-time deployment. Flow Matching offers an efficient alternative by transporting noisy speech toward clean speech through an ordinary differential equation with few function evaluations. In this work, we propose a skip-free encoder-decoder backbone for flow-matching speech enhancement, guided by Latent Representation Alignment (LRA). Instead of relying on U-Net skip connections, which may transfer noise-correlated low-level features to the decoder, the proposed model aligns its bottleneck and decoder representations with clean latent features extracted from a frozen Descript Audio Codec encoder-decoder without quantization. This codec-aligned supervision promotes compact clean-speech representations while preserving efficient few-step inference. Experiments on WSJ0-CHiME3 and VoiceBank-DEMAND show improved PESQ and perceptual quality, especially on VoiceBank-DEMAND, using only five function evaluations.