Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note2 min read

Poster: Exploring the Limits of Audio-Based Detection of Turkish Phone Call Scams

Source

arXiv

arxiv.org/abs/2606.24523v1

Read the source

Source

Plain-Language Brief

Researchers introduce a new dataset of Turkish phone call conversations to test large language models' ability to detect scams. They find that using corrected or uncorrected transcripts improves results compared to processing raw audio.

What the Document Says

  • The research focuses on exploring the use of large language models (LLMs) for detecting phone scams in Turkish, a low-resource language.
  • They create a new dataset containing 100 aligned audio-transcript pairs of scam and benign conversations in Turkish.
  • Seven LLMs from three model families are evaluated under three input conditions: raw audio, automatic speech-to-text transcripts, and transcripts refined by a native speaker.
  • The results indicate that transcript-based inputs perform better than direct audio processing.
  • Human-corrected and uncorrected transcripts produce comparable results.

Key Concepts Mentioned

  • scam phone calls
  • Turkish language
  • large language models
  • dataset
  • scam detection
  • transcript
  • speech-to-text

Claims Made by the Document

  • Scam detection in Turkish is difficult due to limited technological defenses and scarce annotated data.
  • Transcript-based inputs outperform direct audio processing for scam detection in Turkish.

Evidence Used

  • The researchers introduce a new dataset for Turkish scam conversations
  • Results from evaluating seven LLMs under three input conditions

Methods / Approach

  • Creating a multi-modal dataset of Turkish phone call conversations
  • Evaluating seven LLMs under different input conditions

Limitations Stated by the Authors

The study focuses on Turkish, which may limit its applicability to other low-resource languages.

Limitations Noted by the Agent

Agent interpretation — not from the source.

The findings are based on a single dataset and seven specific models. Other datasets or models may yield different results.

Open Questions

  • How do other LLMs perform for scam detection in Turkish?
  • Can the findings be replicated using other low-resource languages?

Notes for Future Q&A

May include interpretation. Labeled where it does.

The research highlights the need for further study on AI safety and fraud prevention, particularly in culturally and linguistically diverse settings.

Raw Abstract / Excerpt

Scam phone calls exploit vulnerable communities worldwide, yet research on detection has focused almost exclusively on English and other high-resource languages. In low-resource settings such as Turkish, detection is especially difficult, as annotated data is scarce and technological defenses remain limited. This research investigates how large language models (LLMs) can support scam detection in Turkish by introducing the first public multi-modal dataset of 100 aligned audio-transcript pairs of scam and benign conversations. We evaluate seven LLMs spanning three model families: Gemini 2.5 (Flash, Flash-Lite, Pro), GPT-4o, and Qwen (Max, Plus, Turbo), under three input conditions: raw audio, automatic speech-to-text transcripts, and transcripts refined by a native speaker. Our results suggest that transcript-based inputs consistently outperform direct audio processing, while human-corrected and uncorrected transcripts perform comparably. By centering a low-resource language and real world threat, this work highlights the urgent need for culturally and linguistically inclusive AI safety research and more robust multi-modal systems for fraud prevention.