Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

From Voting to Agent Collaboration: Answer-Type-Aware LLM Pipelines for BioASQ 14b

Source

arXiv

arxiv.org/abs/2607.06452v1

Read the source

Source

Plain-Language Brief

Researchers have developed a system for answering biomedical questions using large language models (LLMs). The system is designed to improve the robustness of answers by selecting different inference procedures for different types of questions: yes/no, factoid, and list questions. For yes/no questions, the system uses snippet shuffling and self-reflection to make decisions more stable. For factoid questions, it combines full-snippet input with chain-of-thought-based in-context learning for accurate biomedical entity identification. For list questions, a multi-agent architecture is employed for evidence extraction, candidate generation, answer verification, and final aggregation. The system was tested on BioASQ 13b and BioASQ 14b Task B challenges and showed competitive performance.

What the Document Says

  • The study presents a question-type-specific LLM framework for improving answer robustness and evidence grounding in biomedical question answering.
  • For yes/no questions, snippet shuffling and self-reflection are used to reduce sensitivity to evidence ordering and improve decision stability.
  • For factoid questions, full-snippet input is combined with chain-of-thought-based in-context learning to support accurate biomedical entity identification.
  • For list questions, a multi-agent architecture is employed for evidence extraction, candidate generation, answer verification, and final aggregation.
  • The framework was evaluated on BioASQ 13b and BioASQ 14b Task B challenges.

Key Concepts Mentioned

  • large language model
  • biomedical question answering
  • yes/no questions
  • factoid questions
  • list questions
  • snippet shuffling
  • self-reflection
  • chain-of-thought
  • multi-agent architecture

Claims Made by the Document

  • The framework improves answer robustness and evidence grounding.
  • The system shows competitive performance on BioASQ challenges.

Evidence Used

  • Preliminary experiments on BioASQ 13b were used to identify effective inference strategies for each question type, and the resulting framework was subsequently evaluated in the official BioASQ 14b Task B challenge.
  • In the official evaluation, our framework showed competitive performance across multiple batches and achieved first place in the factoid subtask of Batch 4.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Biomedical question answering requires not only accurate extraction of information from scientific literature but also reliable integration of evidence across multiple documents. This study presents a question-type-specific large language model (LLM) framework for BioASQ 14b Task B, designed to improve answer robustness and evidence grounding in biomedical question answering. Rather than applying a single prompting strategy to all questions, the framework selects different inference procedures for yes/no, factoid, and list questions according to their distinct reasoning and evaluation requirements. For yes/no questions, snippet shuffling and self-reflection are used to reduce sensitivity to evidence ordering and improve decision stability. For factoid questions, full-snippet input is combined with chain-of-thought-based in-context learning to support accurate biomedical entity identification. For list questions, a multi-agent architecture is employed, in which evidence extraction, candidate generation, answer verification, and final aggregation are handled collaboratively. Preliminary experiments on BioASQ 13b were used to identify effective inference strategies for each question type, and the resulting framework was subsequently evaluated in the official BioASQ 14b Task B challenge. In the official evaluation, our framework showed competitive performance across multiple batches and achieved first place in the factoid subtask of Batch 4. These results demonstrate the effectiveness of combining question-type-specific inference, ensemble prediction, and agent-based verification for reliable biomedical question answering.