conversational agents
6 notes, newest first.
Benchmarking the Benchmarks: Evaluating Benchmarks for Conversational Agents
Researchers introduce a framework to evaluate the quality of benchmarks used for task-oriented conversational agents, which can lead to unreliable evaluations due to inconsistencies, simplistic scenarios, or limited…
Benchmarking the Benchmarks: Evaluating Benchmarks for Conversational Agents
Researchers have developed a new method to assess the quality of benchmarks used to evaluate task-oriented conversational agents. The method uses artificial intelligence judges to determine if benchmarks are…
Benchmarking the Benchmarks: Evaluating Benchmarks for Conversational Agents
Researchers have introduced a new framework to evaluate the quality of benchmarks used for task-oriented conversational agents. The framework uses language models (LLMs) to assess the consistency, complexity, and…
Benchmarking the Benchmarks: Evaluating Benchmarks for Conversational Agents
The paper introduces a reference-free framework to evaluate the quality of task-oriented conversational agent benchmarks using LLM (Large Language Model) judges. The framework assesses consistency, complexity, and…
Benchmarking the Benchmarks: Evaluating Benchmarks for Conversational Agents
The authors propose a reference-free framework to evaluate the quality of task-oriented conversational agent benchmarks, which are often used to assess these agents. The framework uses Large Language Models (LLMs) as…
Behavior-Adaptive Conversational Agents: Toward a Fluid Personality Framework
The authors propose a Fluid Personality Framework for conversational agents to adapt their metaphorical personas and personality expression levels based on the task context, user goals, and situational urgency.
