The complexities of patient-centred conversational artificial intelligence

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: João Matos, Olivia Buege, Donny Cheung, Gary S. Collins, Paula Dhiman, Nan Li, Bingyu Mao, Benjamin W. Nelson, Michail Ouroutzoglou, Paul Varghese, Jonathan Amar
• Venue: arXiv: cs.AI
• Published: 2026-07-09
• Collected: 2026-07-10
• URL: http://arxiv.org/abs/2607.08625v1
• PDF: https://arxiv.org/pdf/2607.08625v1
• DOI: UNKNOWN

Plain-Language Brief

Researchers analyzed 2,053 conversations between patients and AI chatbots powered by large language models (LLMs) for symptom assessment. They found that communication patterns and emotional expressions vary widely among users. To test the realism of a patient simulator, they compared it to real conversations with 15 human graders, who were almost unable to distinguish between the two. The researchers also evaluated four LLMs in urgency assessment using five distinct patient personas across 1,164 clinician-graded cases. They discovered that communication style can significantly impact triage outcomes. The study suggests that AI systems for healthcare conversations need to account for communication diversity, as those designed for idealized interactions might underperform and worsen health disparities in the real world.

What the Document Says

• Researchers analyzed 2053 conversations between patients and AI chatbots powered by large language models (LLMs) for symptom assessment.
• Communication patterns and emotional expressions vary widely across users in these conversations.
• The researchers developed a patient simulator that separates clinical content, emotional state, conversational strategy, and communication style.
• In a Turing-inspired evaluation with 15 human graders, the simulated conversations were nearly indistinguishable from real ones, with an accuracy of 55%.
• The researchers used five distinct patient personae to evaluate the performance of four LLMs in urgency assessment.
• Communication style can significantly alter triage outcomes according to this study.
• AI systems for healthcare conversations need to account for communication diversity to avoid underperforming and worsening health disparities.

Key Concepts Mentioned

• Patient-centered conversational AI
• Large language models (LLMs)
• Symptom assessment
• Communication patterns
• Emotional expressions
• Patient simulator
• Turing-inspired evaluation
• Urgency assessment
• Communication style
• Triage outcomes
• Health disparities

Claims Made by the Document

• AI systems for healthcare conversations need to account for communication diversity.
• The developed patient simulator can mimic real patient-chatbot conversations.
• Communication style significantly impacts triage outcomes.

Evidence Used

• Analysis of 2053 conversations between patients and AI chatbots
• Development and use of a patient simulator
• Turing-inspired evaluation with 15 human graders
• Evaluation of four LLMs using five distinct patient personas across 1164 cases

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Consumer-facing health chatbots powered by large language models (LLMs) are increasingly used for symptom assessment. However, chatbot development and evaluation often rely on cooperative, articulate, simulated patients. We analysed 2,053 real patient-chatbot conversations and found that communication patterns and expression of emotions vary widely across users. We developed a patient simulator that separately models clinical content, emotional state, conversational strategy, and communication style. In a Turing-inspired evaluation of realism with 15 human graders, simulated conversations were nearly indistinguishable from real ones, with human graders achieving an accuracy of 55%. We used five distinct patient personae, across 1,164 clinician-graded cases, to evaluate the performance of four LLMs in urgency assessment. We found that communication style can significantly alter triage outcomes. Patient-centred conversational artificial intelligence must accommodate communication diversity: systems designed for idealised, rather than realistic, interactions risk underperforming and amplifying health disparities when deployed in the real world.