When Synthetic Speech Is All You Have: Better Call GRPO

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Shashi Kumar, Yanis Labrak, Hasindri Watawana, Sergio Burdisso, Esaú Villatoro-Tello, Kadri Hacioğlu, Petr Motlicek, Andreas Stolcke
• Venue: arXiv: cs.CL
• Published: 2026-07-09
• Collected: 2026-07-10
• URL: http://arxiv.org/abs/2607.08409v1
• PDF: https://arxiv.org/pdf/2607.08409v1
• DOI: UNKNOWN

Plain-Language Brief

Researchers present a method called Group Relative Policy Optimization (GRPO) that improves the accuracy of synthetic speech-to-text conversion compared to traditional methods. This could be particularly useful in regulated domains such as banking where collecting real speech is costly and privacy-sensitive.

What the Document Says

• The research focuses on adapting Language Model (LLM) based Automatic Speech Recognition (ASR) for regulated domains like banking, where real speech is expensive and legally restricted to collect. The solution proposed is synthetic text-to-speech (TTS) as a substitute.
• The challenge lies in the acoustic mismatch between synthetic speech and real recordings. Most work in this area has been within supervised fine-tuning (SFT).
• The authors propose using Reinforcement Learning instead, and show that GRPO extracts more from synthetic speech than SFT.
• GRPO reduces Word Error Rate (WER) by 40% relative to SFT (36.71% → 22.09%).
• A combination of SFT followed by GRPO further reduces WER by 45%.
• The improvement is attributed to behavior rather than representation, with GRPO reducing insertion errors and improving speech-to-text alignment by better anchoring attention to audio while leaving early-layer representations intact.
• The authors suggest that in situations where synthetic speech is the main resource, Reinforcement Learning should be preferred over Supervised Fine-Tuning.

Key Concepts Mentioned

• Automatic Speech Recognition (ASR)
• Language Model (LLM)
• synthetic text-to-speech (TTS)
• Reinforcement Learning
• Group Relative Policy Optimization (GRPO)
• Word Error Rate (WER)

Claims Made by the Document

• GRPO extracts more from synthetic speech than SFT.
• GRPO reduces WER by 40% relative to SFT.
• A combination of SFT followed by GRPO further reduces WER by 45%.

Evidence Used

• SYNTHETIC-ONLY ADAPTATION OF THE MODEL WITH GRPO... REDUCES WER BY 40% RELATIVE TO SFT (36.71%→22.09%)

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

LLM-based ASR adapted to regulated domains such as banking is bottlenecked by privacy: real speech is costly and legally constrained to collect, making synthetic text-to-speech (TTS) an attractive substitute. Yet synthetic speech stays acoustically mismatched with real recordings, and work on this gap has stayed within supervised fine-tuning (SFT). We instead turn to reinforcement learning, and show that Group Relative Policy Optimization (GRPO) extracts far more from the same synthetic speech than SFT. Synthetic-only adaptation of the model with GRPO, a critic-free method rewarding low-WER hypotheses, reduces WER by 40% relative to SFT (36.71%$\to$22.09%), and an SFT-then-GRPO combination pushes this further to 45%. We trace the gain to behavior rather than representation: GRPO reduces insertion errors by improving stopping calibration and speech-to-text alignment by better anchoring attention to audio, leaving early-layer representations intact. When synthetic speech is the main resource, reinforcement learning should be preferred over supervised fine-tuning.