Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

Beyond Top-K: Replacing Black-Box Retrieval with Interpretable Agentic Operations

Source

arXiv

arxiv.org/abs/2608.06305v1

Read the source

Source

Plain-Language Brief

The authors propose a new method for document search called READ (Reliable Embedding-free Agentic Document-search). This method uses an interpretable agent that performs three operations: normalized lexical search, structural navigation, and bounded span reads. The agent is designed to address the issues with current retrieval-augmented generation systems on specific types of documents like financial statements, audit reports, and regulatory returns. The authors compare READ with a top-k tool and show that READ performs significantly better in answering verified questions.

What the Document Says

  • The authors propose a new method called READ (Reliable Embedding-free Agentic Document-search).
  • READ consists of three deterministic operations: normalized lexical search, structural navigation, and bounded span reads.
  • These operations are exposed over the Model Context Protocol, allowing for replayable audit trails.
  • The authors compare READ with a top-k tool on 51 verified questions.
  • READ answers 58.8% of the questions correctly compared to 15.7% by dense retrieval (p_Holm = 2 x 10^-5).
  • When given the same loop but a top-k tool, the agent reaches only 27.5%.
  • The authors also report that BM25 is statistically indistinguishable from READ, indicating that the result separates embedding-based from embedding-free retrieval, not agentic from lexical search.

Key Concepts Mentioned

  • document search
  • RETRO augmentation
  • agentic operations
  • financial statements
  • audit reports
  • regulatory returns
  • BM25

Claims Made by the Document

  • READ outperforms dense retrieval in answering verified questions.
  • The top-k tool performs worse than READ when given the same loop.

Evidence Used

  • On 51 verified questions READ answers 58.8% against dense retrieval's 15.7% (p_Holm = 2 x 10^-5)

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Retrieval-augmented generation over long documents is dominated by one design: chunk the text, embed the chunks, and surface the top-k nearest neighbours of the query. We argue that for an important class of documents -- financial statements, audit reports, regulatory returns -- this design is structurally unsound, and we make the argument measurable. On a 780-page government financial report, 86.8% of content lines are table rows, thousands of near-identical figures compete in one embedding space, and a figure inherits its unit from a header a median of 13 lines above it -- so a chunk boundary routinely separates a number from whether it is in lakh or crore, an error of two orders of magnitude. A table-aware chunker built as a steelman fixes the unit problem but leaves 27-30% of numeric chunks with no fiscal-year header at every chunk size we tried. We propose READ (Reliable Embedding-free Agentic Document-search), in which an agent reads the raw document through three deterministic operations -- normalized lexical search, structural navigation, and bounded span reads -- exposed over the Model Context Protocol, so a trajectory is a replayable audit trail, not an opaque similarity score. On 51 verified questions READ answers 58.8% against dense retrieval's 15.7% (p_Holm = 2 x 10^-5) -- or 35.3% tuned, which READ still leads by 23.5 points (p_Holm = 0.017). An agent given the same loop but a top-k tool reaches only 27.5%, locating the gain in the interface rather than in iteration. We also report what the evidence does not support: BM25 is statistically indistinguishable from READ, so our result separates embedding-based from embedding-free retrieval, not agentic from lexical search.