Self-prompting and cross-model consensus enable reproducible data extraction from scientific literature with large language models

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Valentin Romanov, Monique Bax, Steven Niederer
• Venue: arXiv: cs.AI
• Published: 2026-08-19
• Collected: 2026-08-20
• URL: http://arxiv.org/abs/2608.19025v1
• PDF: https://arxiv.org/pdf/2608.19025v1
• DOI: UNKNOWN
• Registry group: academic_apis_and_preprints
• Evidence role: canonical_academic_index
• Language: en
• Country / region: UNKNOWN

Plain-Language Brief

The study explores the ability of large language models (LLMs) to extract contextualized information from research articles efficiently. The authors present four workflows that demonstrate LLMs can perform well in data extraction but may struggle with scientific context and nuance. They also discuss the challenges in autonomous literature discovery, and suggest a division of labor where experts set evidence standards, models cross-check extractions, and humans resolve disputes, aiming to scale scientific data curation without compromising expert oversight.

What the Document Says

• The text discusses the challenging task of extracting nuanced and contextualized information from research articles.
• It presents four workflows using large language models (LLMs) for this purpose.
• In the first workflow, LLMs perform well in data extraction but struggle with interpreting scientific context and nuance when given an expert-curated prompt.
• In the second workflow, LLMs can write their own prompts that are almost as effective as expert-written ones for data extraction.
• The third workflow discusses the difficulty of autonomous discovery of research literature using LLMs, which may miss or hallucinate references.
• In the fourth workflow, LLMs can create new datasets from published guidelines that closely match human-expert judges, but still require a human-in-the-loop.
• The authors propose an auditable division of labor where experts set evidence standards, models cross-check extractions, and humans resolve disputes to scale scientific data curation while maintaining expert oversight.

Key Concepts Mentioned

• large language models
• scientific literature
• data extraction
• workflows
• expert oversight

Claims Made by the Document

• LLMs can perform well in data extraction.
• LLMs may struggle with interpreting scientific context and nuance when given an expert-curated prompt.
• LLMs can write their own prompts that are almost as effective as expert-written ones for data extraction.
• Autonomous literature discovery using LLMs is difficult.
• LLMs can create new datasets from published guidelines.
• A division of labor between experts, models, and humans can help scale scientific data curation while maintaining expert oversight.

Evidence Used

• The text describes the performance of LLMs in data extraction based on four workflows presented.
• It details the challenges faced during autonomous literature discovery using LLMs.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Accurately extracting nuanced, contextualized data from research articles is laborious and time intensive. Here, we investigate the performance of frontier, browser-based large language models (LLMs) to extract highly contextualized information. We demonstrate four escalating workflows, 1) given an expert curated prompt and research articles, most frontier LLMs perform well at data extraction, however can struggle with interpreting scientific context and nuance, 2) given simple instructions, LLMs can author their own prompts which were almost as eNective as expert-written prompts, 3) autonomous discovery of research literature was diNicult, agents either missed or hallucinated references, and 4) LLMs can create new datasets from published guidelines that closely match human-expert judges, but still require a human-in-the-loop. Together, these findings define an auditable division of labour in which experts specify the evidence standard, models cross-check repeated extractions and researchers resolve disputed cases, providing a practical route to scaling scientific data curation without relinquishing expert oversight.