FIELD NOTES TOPIC
benchmark quality
2 notes, newest first.
Benchmarking the Benchmarks: Evaluating Benchmarks for Conversational Agents
Researchers have developed a new method to assess the quality of benchmarks used to evaluate task-oriented conversational agents. The method uses artificial intelligence judges to determine if benchmarks are…
Benchmarking the Benchmarks: Evaluating Benchmarks for Conversational Agents
The authors propose a reference-free framework to evaluate the quality of task-oriented conversational agent benchmarks, which are often used to assess these agents. The framework uses Large Language Models (LLMs) as…
