Source note2 min read
Exploring the Semantic Gap in Agentic Data Systems: A Formative Study of Operationalization Failures in Analytical Workflows
Source
arXiv
arxiv.org/abs/2607.00828v1
Source
- Source name: arXiv
- Source kind: paper
- Authors / organization: Jalal Mahmud, Eser Kandogan
- Venue: arXiv: cs.DB
- Published: 2026-07-01
- Collected: 2026-07-02
- URL: http://arxiv.org/abs/2607.00828v1
- PDF: https://arxiv.org/pdf/2607.00828v1
- DOI: UNKNOWN
Plain-Language Brief
A study identifies common failures in generating and executing analytical workflows using large language models across finance, human resources, and public safety domains. The authors found five recurring classes of failures: comparative grounding, process reasoning, quantitative reasoning, role confusion, and policy grounding. These findings suggest a semantic gap between user-level analytical concepts and the information available to workflow-generation systems.
What the Document Says
- The text discusses the use of large language models (LLMs) in generating queries, invoking tools, and constructing analytical workflows.
- Despite improvements in workflow generation and execution, there is often a semantic gap between the required information for operationalizing analytical concepts and what is explicitly represented in database schemas and data values.
- The authors conducted a cross-domain study of operationalization failures in agent-generated analytical workflows across finance, human resources, and public safety domains.
- Across 236 analytical intents, the study identified 153 recurring failures despite successful workflow generation and execution.
- The authors categorized these failures into five classes: comparative grounding, process reasoning, quantitative reasoning, role confusion, and policy grounding.
- The findings suggest a semantic gap between user-level analytical concepts and the information available to workflow-generation systems.
Key Concepts Mentioned
- large language models
- analytical workflows
- semantic gap
- operationalization failures
- comparative grounding
- process reasoning
- quantitative reasoning
- role confusion
- policy grounding
Claims Made by the Document
- There is a semantic gap between user-level analytical concepts and the information available to workflow-generation systems.
- The authors identified recurring operationalization failures in agent-generated analytical workflows.
Evidence Used
- The study identifies 153 recurring failures across various domains.
Methods / Approach
UNKNOWN
Limitations Stated by the Authors
UNKNOWN
Limitations Noted by the Agent
Agent interpretation — not from the source.
UNKNOWN
Open Questions
UNKNOWN
Notes for Future Q&A
May include interpretation. Labeled where it does.
UNKNOWN
Raw Abstract / Excerpt
Large language models (LLMs) are increasingly used to generate queries, invoke tools, and construct analytical workflows. Although recent advances have substantially improved workflow generation and execution, the semantic information required to operationalize analytical concepts often lies beyond what is explicitly represented in database schemas and data values. We present a cross-domain formative study of operationalization failures in agent-generated analytical workflows. Across 236 analytical intents spanning finance, human resources, and public safety domains, we identify 153 recurring failures despite successful workflow generation and execution. Our analysis reveals five recurring classes of failures: comparative grounding, process reasoning, quantitative reasoning, role confusion, and policy grounding. These findings suggest a semantic gap between user-level analytical concepts and the information available to workflow-generation systems. More broadly, they raise questions about the admissibility of analytical operations and suggest that future agentic data systems may require richer semantic representations to bridge the gap between analytical intent and executable computation.
