KGCaRe: Explainable Complex Conditional Question Answering using Automatic Knowledge Graph Construction and Context Retrieval with LLMs

Source

Source name: arXiv
Source kind: paper
Authors / organization: Ghanshyam Verma, Simanta Sarkar, Devishree Pillai, Hotaka Shiokawa, Yourong Xu, Fiona Veazey, Peter Hubbert, Hui Su, Paul Buitelaar
Venue: arXiv: cs.CL
Published: 2026-08-10
Collected: 2026-08-11
URL: http://arxiv.org/abs/2608.09779v1
PDF: https://arxiv.org/pdf/2608.09779v1
DOI: UNKNOWN
Registry group: academic_apis_and_preprints
Evidence role: canonical_academic_index
Language: en
Country / region: UNKNOWN

Plain-Language Brief

The study proposes a new method called KGCaRe for answering complex conditional questions using large language models (LLMs) and retrieval-augmented generation (RAG). This approach combines neural retrieval with symbolic reasoning over LLM-generated knowledge graphs. The researchers constructed a knowledge graph from documents using a multi-prompt extraction strategy, embedded the documents into a vector store for neural retrieval, and used iterative graph traversal guided by the LLM to extract relevant information. They evaluated KGCaRe on two complex conditional QA datasets and found it outperformed existing baselines across multiple LLMs.

What the Document Says

The study presents a method named KGCaRe for addressing complex conditional question answering using LLMs and RAG.
KGCaRe combines neural retrieval with symbolic reasoning over LLM-generated knowledge graphs.
It constructs a knowledge graph from documents using multi-prompt extraction and stores it in a graph database.
Simultaneously, the documents are embedded into a vector store for neural retrieval.
KGCaRe performs iterative graph traversal guided by the LLM to extract relevant triples and prune irrelevant information.
Additional clue entities are used if the initial traversal does not provide satisfactory context.
Relevant triples, along with semantically retrieved text passages, are then used to generate answers with explanations.
Evaluation was done on two complex conditional QA datasets, and KGCaRe outperformed existing baselines across multiple LLMs such as Mistral, Mixtral, GPT-3.5, and GPT-4o.

Key Concepts Mentioned

KGCaRe
Large Language Models (LLMs)
Retrieval-Augmented Generation (RAG)
knowledge graphs (KGs)
complex conditional question answering

Claims Made by the Document

Improving reasoning and answer accuracy for complex conditional question answering by augmenting RAG with unstructured and structured knowledge.
Outperforming existing baselines across multiple LLMs.

Evidence Used

Results on two complex conditional QA datasets showing KGCaRe's superior performance compared to existing baselines.
List of outperformed baselines: Vanilla LLM, Code Prompt, Text Prompt, Think-on-Graph, Vanilla RAG, and HybridContextQA.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Answering complex conditional questions using Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) remains a challenge, particularly in domain-specific contexts where general-purpose LLMs and RAG tend to underperform. We hypothesize that augmenting RAG with unstructured and structured knowledge, extracted from both documents and knowledge graphs (KGs), can improve reasoning and answer accuracy for such tasks. To test this, we propose KGCaRe, a hybrid approach that combines neural retrieval with symbolic reasoning over LLM-generated KGs. KGCaRe constructs a KG from documents using a multi-prompt extraction strategy and stores it in a graph database. Simultaneously, the documents are embedded into a vector store to enable neural retrieval. KGCaRe performs innovative iterative graph traversal guided by the LLM to extract relevant triples, prune irrelevant information, and uses additional clue entities to traverse the graph again if the initial traversal does not provide satisfactory context to generate the answer. The relevant triples extracted from the KG in path form, along with semantically retrieved text passages, are then fed into custom KGCaRe prompts to generate answers to the complex conditional questions with explanations. We evaluate KGCaRe on two complex conditional QA datasets. Our results on these datasets show that KGCaRe consistently outperforms existing baselines, including Vanilla LLM, Code Prompt, Text Prompt, Think-on-Graph, Vanilla RAG, and HybridContextQA, across multiple LLMs such as Mistral, Mixtral, GPT-3.5, and GPT-4o. We publicly release the software pipeline that we developed to implement the proposed KGCaRe approach.