DuplexWorld: Can voice agents help you get through the day?

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Aryan Vijay Bhosale, Harshit Rajgarhia, Akhil Pothanapalli, Asif Shaik, Abhishek Mukherji, Dinesh Manocha
• Venue: arXiv: cs.SD
• Published: 2026-08-11
• Collected: 2026-08-12
• URL: http://arxiv.org/abs/2608.10716v1
• PDF: https://arxiv.org/pdf/2608.10716v1
• DOI: UNKNOWN
• Registry group: academic_apis_and_preprints
• Evidence role: canonical_academic_index
• Language: en
• Country / region: UNKNOWN

Plain-Language Brief

The research paper DuplexWorld evaluates voice agents' performance in various mundane activities like banking, insurance, travel, healthcare, logistics, and pathfinding. The evaluation includes 156 scenarios testing conversational and analytical capabilities across 3 axes: Pass@1, turn-taking, and DNSMOS. The results show that even the best voice agents have significant room for improvement on all 3 axes.

What the Document Says

• The research evaluates voice agents' performance in various mundane activities through six worlds: banking, insurance, travel, healthcare and logistics, and Pathfinding.
• Voice agents are evaluated based on eleven different types of conversations across 156 scenarios, totaling more than 350 hours of conversation.
• The evaluation includes agentic, conversational, and speech-naturalness metrics.
• The results show that even the best voice agents leave substantial room for improvement in all three axes: Pass@1 (0.490), turn-taking (0.653), and DNSMOS (3.378).

Key Concepts Mentioned

• voice agents
• S2S voice agents
• evaluation
• benchmarks
• DuplexWorld
• banking
• insurance
• travel
• healthcare
• logistics
• Pathfinding
• conversational dialogue
• conversational and analytical capability
• agentic tool calling
• database
• scenarios
• metrices
• Pass@1
• turn-taking
• DNSMOS

Claims Made by the Document

• Existing benchmarks fail to holistically evaluate voice agents.
• Voice agents can assist in various mundane activities beyond database manipulation.

Evidence Used

• The introduction of six worlds and extensive evaluation comprising agentic, conversational, and speech-naturalness metrics as stated by the authors.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Speech-to-speech (S2S) voice agents are increasingly being incorporated into enterprise for customer care and as daily companions for consumers owing to the ease of the conversational modality over text. However, existing benchmarks fail to holistically evaluate voice agents along axes that really matter and are shaped as tests of agentic tool calling against a database. We believe they fail to adequately account for the diversity of conversational dialogue that mundane activities introduce and further, never test how faithfully an agent can assist on tasks that move beyond database manipulation. To tackle this DuplexWorld introduces six worlds where voice agents are especially useful: banking, insurance, travel, healthcare and logistics, and Pathfinding. Agents are evaluated on eleven different types of conversations across 156 scenarios (350+ hours of conversation), each testing conversational and analytical capability to varying degrees. Through extensive evaluation comprising agentic, conversational and speech-naturalness metrics, we show that even the best voice agents leave substantial room for improvement on all 3 axes (Pass@1: 0.490, turn-taking: 0.653, DNSMOS: 3.378). We perform extensive analysis on agentic v conversational performance, world- and conversation type-wise performance, failure modes exploring the explore v exploit lens for Pathfinding conversations and voice agent reliability over all six worlds.