Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

From Proprietary to Open-Source: Bridging the Distribution Gap via Multi-Agent Protocol Distillation in Agentic Search

Source

arXiv

arxiv.org/abs/2607.24280v1

Read the source

Source

Plain-Language Brief

Researchers have proposed a new method called Multi-Agent Protocol Distillation (MAPD) to improve the performance of large language models in solving knowledge-intensive tasks. This method addresses the challenge of sparse supervision by optimizing with outcome-based reinforcement learning and bridging the distribution gap between proprietary models and open-source models. MAPD uses a structured, style-normalized protocol as an intermediate representation, which is provided to a privileged branch of the student policy during training. The framework was tested on seven QA benchmarks and outperformed competitive distillation and RL methods.

What the Document Says

  • The paper proposes Multi-Agent Protocol Distillation (MAPD) for improving large language models in solving knowledge-intensive tasks.
  • MAPD addresses the sparse supervision challenge with outcome-based reinforcement learning.
  • Proprietary models are used as teachers due to their strong reasoning capabilities.
  • Conventional logit-matching is not applicable due to hidden logits and mismatched tokenizers.
  • Raw natural language trajectory imitation transfers superficial stylistic artifacts instead of core reasoning competence.
  • MAPD uses a structured, style-normalized protocol as an intermediate representation.
  • An offline multi-agent system decomposes each query, retrieves supporting evidence, repairs failed searches, and converts the resulting exploration trace into a JSON protocol.
  • During training, the protocol is provided to a privileged branch of the student policy.
  • The framework outperformed competitive distillation and RL methods in extensive evaluations across seven QA benchmarks.
  • MAPD achieved average success rates of 39.4% on Qwen3-1.7B and 44.4% on Qwen3-4B.
  • The framework generalizes robustly across diverse proprietary teachers.

Key Concepts Mentioned

  • large language models
  • knowledge-intensive tasks
  • outcome-based reinforcement learning
  • knowledge distillation
  • proprietary models
  • multi-agent protocol distillation
  • QA benchmarks

Claims Made by the Document

  • MAPD improves large language models in solving knowledge-intensive tasks.
  • MAPD outperforms competitive distillation and RL methods.

Evidence Used

  • Extensive evaluations across seven QA benchmarks demonstrate that MAPD consistently outperforms competitive distillation and RL, achieving average success rates of 39.4% on Qwen3-1.7B and 44.4% on Qwen3-4B.

Methods / Approach

  • Multi-Agent Protocol Distillation (MAPD)
  • outcome-based reinforcement learning
  • offline multi-agent system

Limitations Stated by the Authors

  • UNKNOWN: The paper does not state any specific limitations of the proposed method.

Limitations Noted by the Agent

Agent interpretation — not from the source.

  • The method requires access to proprietary models, which may be unavailable or costly.
  • The performance of MAPD might degrade if the quality of the supporting evidence retrieved by the offline multi-agent system is poor.

Open Questions

  • What are the specific components of the offline multi-agent system?
  • How does the framework handle queries that cannot be decomposed or repaired?

Notes for Future Q&A

May include interpretation. Labeled where it does.

  • The paper focuses on improving large language models for knowledge-intensive tasks using a new method called Multi-Agent Protocol Distillation (MAPD). The proposed framework addresses the sparse supervision challenge and outperforms competitive distillation and RL methods in extensive evaluations.
  • The offline multi-agent system is used to decompose each query, retrieve supporting evidence, repair failed searches, and convert the resulting exploration trace into a JSON protocol.

Raw Abstract / Excerpt

Agentic search enables large language models to solve knowledge-intensive tasks by interleaving multi-step reasoning with retrieval, yet optimizing this with outcome-based reinforcement learning (RL) provides only sparse supervision. Knowledge distillation can supply denser guidance, and advanced proprietary models with their strong reasoning capabilities are promising teachers. While distilling from proprietary models can densify this supervisory signal, conventional logit-matching is precluded by hidden logits and mismatched tokenizers, whereas raw natural language trajectory imitation transfers superficial stylistic artifacts rather than core reasoning competence. To address the heterogeneous distillation problem and bridge the distribution gap, we propose Multi-Agent Protocol Distillation (MAPD), a joint distillation and RL framework uses a structured, style-normalized protocol as an intermediate representation. An offline multi-agent system (MAS) decomposes each query, retrieves supporting evidence, repairs failed searches, and converts the resulting exploration trace into a JSON protocol containing the task type, reasoning plan, and extractive grounding facts. During training, the protocol is provided only to a privileged branch of the student policy, whose token distributions furnish a dense distillation signal alongside the sparse RL objective. Extensive evaluations across seven QA benchmarks demonstrate that MAPD consistently outperforms competitive distillation and RL, achieving average success rates of 39.4% on Qwen3-1.7B and 44.4% on Qwen3-4B. Crucially, the framework generalizes robustly across diverse proprietary teachers while effectively mitigating the student policy from style drift and verbosity degeneration.