Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note2 min read

From Voting to Agent Collaboration: Answer-Type-Aware LLM Pipelines for BioASQ 14b

Source

arXiv

arxiv.org/abs/2607.06452v1

Read the source

Source

Plain-Language Brief

Researchers present a framework using question-type-specific large language models (LLMs) to improve answer robustness and evidence grounding in biomedical question answering. The framework uses different inference procedures for yes/no, factoid, and list questions, with the goal of reducing sensitivity to evidence ordering and improving decision stability. It was evaluated in the BioASQ 14b Task B challenge and showed competitive performance across multiple batches and first place in the factoid subtask of Batch 4.

What the Document Says

  • The study presents a question-type-specific LLM framework for BioASQ 14b Task B.
  • For yes/no questions, snippet shuffling and self-reflection are used to improve decision stability.
  • For factoid questions, full-snippet input and chain-of-thought-based in-context learning are used to support accurate biomedical entity identification.
  • For list questions, a multi-agent architecture is employed for evidence extraction, candidate generation, answer verification, and final aggregation.
  • The framework was evaluated in the official BioASQ 14b Task B challenge.
  • In the evaluation, the framework showed competitive performance across multiple batches and achieved first place in the factoid subtask of Batch 4.

Key Concepts Mentioned

  • Biomedical question answering
  • Large language models
  • Question types
  • Yes/no questions
  • Factoid questions
  • List questions
  • Snippet shuffling
  • Self-reflection
  • Chain-of-thought
  • In-context learning
  • Multi-agent architecture
  • BioASQ
  • Evaluation

Claims Made by the Document

  • The framework improves answer robustness and evidence grounding in biomedical question answering.
  • Snippet shuffling and self-reflection improve decision stability for yes/no questions.

Evidence Used

  • The study presents the framework.
  • The text describes the strategies used for each question type.

Methods / Approach

  • Question-type-specific LLM
  • Snippet shuffling
  • Self-reflection
  • Full-snippet input
  • Chain-of-thought-based in-context learning
  • Multi-agent architecture

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Biomedical question answering requires not only accurate extraction of information from scientific literature but also reliable integration of evidence across multiple documents. This study presents a question-type-specific large language model (LLM) framework for BioASQ 14b Task B, designed to improve answer robustness and evidence grounding in biomedical question answering. Rather than applying a single prompting strategy to all questions, the framework selects different inference procedures for yes/no, factoid, and list questions according to their distinct reasoning and evaluation requirements. For yes/no questions, snippet shuffling and self-reflection are used to reduce sensitivity to evidence ordering and improve decision stability. For factoid questions, full-snippet input is combined with chain-of-thought-based in-context learning to support accurate biomedical entity identification. For list questions, a multi-agent architecture is employed, in which evidence extraction, candidate generation, answer verification, and final aggregation are handled collaboratively. Preliminary experiments on BioASQ 13b were used to identify effective inference strategies for each question type, and the resulting framework was subsequently evaluated in the official BioASQ 14b Task B challenge. In the official evaluation, our framework showed competitive performance across multiple batches and achieved first place in the factoid subtask of Batch 4. These results demonstrate the effectiveness of combining question-type-specific inference, ensemble prediction, and agent-based verification for reliable biomedical question answering.