Comparative Approaches to Agent Retrieval over Large Skill Libraries

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Indivara Kolluru, Nathan Sportsman
• Venue: arXiv: cs.AI
• Published: 2026-08-06
• Collected: 2026-08-07
• URL: http://arxiv.org/abs/2608.06196v1
• PDF: https://arxiv.org/pdf/2608.06196v1
• DOI: UNKNOWN

Plain-Language Brief

The research compares two systems for deciding which skills to load from a large skill library when creating agents. One system combines lexical and dense-embedding retrieval, while the other uses a typed knowledge graph encoding workflow relations. The study found that the hybrid ranker outperforms the graph system in retrieving correct skills in most cases, but the graph can enrich relation semantics.

What the Document Says

• The research focuses on systems for deciding which skills to load from a large skill library for creating agents.
• A hybrid ranker using lexical and dense-embedding retrieval is studied.
• A typed knowledge graph system is also studied, encoding workflow relations like prerequisites, data flow, and ordering.

Key Concepts Mentioned

• Agents
• Skill libraries
• Retrieval systems
• Hybrid ranker
• Typed knowledge graph

Claims Made by the Document

• The hybrid ranker retrieves the correct skill within the top five in 73.5% of cases.
• When used as intended, the graph is significantly worse than the hybrid ranker.

Evidence Used

• On a set of 117 realistic queries, the hybrid ranker outperformed the graph system.
• The graph's performance was -11.2 points lower when used as intended.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Agents backed by large skill libraries must decide which skills to load and in what order. Loading the entire library into context is expensive and provides no structure for autonomous sequencing. We study two systems for this problem over a corpus of 690 skills: a hybrid ranker combining lexical and dense-embedding retrieval for sparse, on-demand loading, and a typed knowledge graph encoding workflow relations such as prerequisites, data flow, and ordering. On a set of 117 realistic, non-echoing queries, the hybrid ranker retrieves the correct skill within the top five in 73.5% +/- 8.0 of cases, leaving roughly a quarter of queries unserved. When used as the design intended (substituting graph neighbours for additional ranked results at matched token budget), the graph is significantly worse (-11.2 points, p = 0.0007). Its LLM-generated edge layer adds nothing over neighbours obtained free from a local embedding pass, and 73% of the queries the ranker misses are not reachable through the graph at all. We attribute this to a pre-filter topology bound. Because the graph's candidate edges are drawn from the same embedding neighbourhood the ranker already searches, 98.6% of typed edges connect skills the ranker had already surfaced together. The graph can enrich relation semantics but cannot extend retrieval reach. We further show that evaluating on author-written queries overstates hit@5 by up to 44 points, which would have hidden these results entirely. Our contribution is a mechanistic account of why added structure does not improve retrieval over a strong ranker, and identify the conditions under which adding structural interdependence into the retrieval is optimal.