FIELD NOTES TOPIC
root cause analysis
3 notes, newest first.
CTBench: Evaluating Troubleshooting Capabilities of AI Agents in Realistic Telecom Network Operations
The paper introduces CTBench, a public benchmark for evaluating AI agents' ability to perform troubleshooting tasks in telecom network operations. The focus is on root cause analysis and path restoration. The authors…
ORCA-bench: How Ready Are Language Model Agents for Oncall?
The study presents ORCA-bench, a benchmark for evaluating the performance of large language models in oncall root cause analysis (RCA). The benchmark exposes a microservice system with six days of data and 1,079 RCA…
OpenRCA 2.0: From Outcome Labels to Causal Process Supervision
The paper introduces OpenRCA 2.0, a new benchmark for root cause analysis (RCA) that includes step-wise causal annotations for large language models (LLMs). The benchmark aims to test the capabilities of LLMs in…
