Source note1 min read
RATED: Reliability-Aware AI Agent Tool Retrieval via Execution Dynamics
Source
arXiv
arxiv.org/abs/2610.13889v1
Source facts
RATED combines a frozen dual encoder for semantic matching with a lightweight contextual-bandit module that reranks tools using real-time execution feedback. The authors introduce EcoAgent, a benchmark of 5,172 tools and more than 210,000 query-tool pairs. They report a 52.2% relative MRR improvement over the ToolLLM API retriever and a reduction in fault-selection rate from 86.2% to 2.9% under a 30% API failure rate, with less than 1 ms of added encode-only latency. Code and data are reported as available.
Analysis
The work makes runtime reliability part of tool retrieval rather than treating availability as an external systems concern. If the benchmark transfers to production tool ecosystems, execution feedback could provide a practical control loop for agents operating under partial failure.
Uncertainty/Doubt
- This is an arXiv preprint. The benchmark construction, reported percentages, code, and CIKM acceptance were not independently reproduced or checked against proceedings here.
