Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

AutoSynthesis: An agentic system for automated meta-analysis

Source

arXiv

arxiv.org/abs/2607.15247v1

Read the source

Source

Plain-Language Brief

AutoSynthesis is a new system designed to automate the process of meta-analysis in various fields like science, medicine, education, and policy. It can take a research question in natural language, perform tasks such as literature search, screening candidate studies, assessing eligibility, extracting quantitative statistics, and computing standardized effect sizes. The system also supports heterogeneity analysis and risk-of-bias assessment. Its results are comparable to those of manual evidence synthesis conducted by experts.

What the Document Says

  • AutoSynthesis is an automated meta-analysis system for various fields
  • It takes a research question in natural language as input
  • It performs tasks such as literature search, screening candidate studies, assessing eligibility, extracting quantitative statistics, and computing standardized effect sizes
  • It supports heterogeneity analysis and risk-of-bias assessment
  • The output is a transparent report aligned with PRISMA guidelines
  • In the application, AutoSynthesis screened over 28 studies and extracted more than 20 quantitative claims
  • The pooled effect estimates produced by AutoSynthesis are similar to Hedges' $g$ of expert-conducted meta-analyses
  • AutoSynthesis supports evidence-based decision-making across disciplines

Key Concepts Mentioned

  • AutoSynthesis
  • meta-analysis
  • research question
  • literature search
  • screening candidate studies
  • assessing eligibility
  • extracting quantitative statistics
  • standardized effect sizes
  • heterogeneity analysis
  • risk-of-bias assessment
  • PRISMA guidelines
  • Hedges' g

Claims Made by the Document

  • AutoSynthesis can make quantitative evidence synthesis more scalable
  • The pooled effect estimates produced by AutoSynthesis are similar to Hedges' $g$ of expert-conducted meta-analyses

Evidence Used

  • screened over 28 studies and extracted more than 20 quantitative claims
  • pooled effect estimates produced by AutoSynthesis are similar to Hedges' g of expert-conducted meta-analyses

Methods / Approach

  • natural language input
  • literature search
  • screening candidate studies
  • assessing eligibility
  • extracting quantitative statistics
  • computing standardized effect sizes
  • heterogeneity analysis
  • risk-of-bias assessment

Limitations Stated by the Authors

  • UNKNOWN
  • UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

  • The study does not provide information on the accuracy or precision of AutoSynthesis compared to manual meta-analysis in all scenarios. More research is needed to fully understand its limitations.

Open Questions

  • What are the potential pitfalls and limitations of AutoSynthesis that were not addressed in this study?

Notes for Future Q&A

May include interpretation. Labeled where it does.

  • The study does not provide detailed information on how AutoSynthesis handles different types of studies or research questions.
  • It would be beneficial to understand if AutoSynthesis can handle complex queries or if it requires simplified questions.

Raw Abstract / Excerpt

Evidence synthesis is crucial for turning primary research into reliable knowledge for science, medicine, education, and policy. Yet, quantitative evidence synthesis remains largely manual and difficult to scale. Here, we introduce AutoSynthesis, an end-to-end multi-agent system for automated meta-analysis. Given a research question in natural language, AutoSynthesis formulates a search strategy, retrieves scientific literature, screens candidate studies, assesses full-text eligibility, extracts quantitative statistics, computes standardized effect sizes, and finally performs random-effects meta-analysis. AutoSynthesis further supports heterogeneity analysis to examine how effect sizes vary across moderators, as well as risk-of-bias assessment. As output, AutoSynthesis produces a transparent report aligned with PRISMA guidelines. In our application, AutoSynthesis screened over 28 studies and extracted more than 20 quantitative claims. The pooled effect estimates produced by AutoSynthesis are similar to Hedges' $g$ of expert-conducted meta-analyses, indicating close agreement with manual evidence synthesis. Together, these results show that AutoSynthesis can make quantitative evidence synthesis more scalable, thereby supporting evidence-based decision-making across disciplines.