oncall

ORCA-bench: How Ready Are Language Model Agents for Oncall?

Source note · Jul 30, 2026

The study presents ORCA-bench, a benchmark for evaluating the performance of large language models in oncall root cause analysis (RCA). The benchmark exposes a microservice system with six days of data and 1,079 RCA…