Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note4 min read

AutoDesign: Meta-Harness Optimization for Long-Horizon Agentic Design

Source

arXiv

arxiv.org/abs/2608.13560v1

Read the source

Source

  • Source name: arXiv
  • Source kind: paper
  • Authors / organization: Yaxin Luo, Haobin Jiang, Jialv Zou, Xu Huang, Wenhao Yan, Haodong Li, Zhengrong Yue, Jing Li, Xiaofu Chen, Xiaohan Zhao, Jiacheng Liu, Jiacheng Cui, Zhiqiang Shen, Xiaotong Li
  • Venue: arXiv: cs.CV
  • Published: 2026-08-13
  • Collected: 2026-08-15
  • URL: http://arxiv.org/abs/2608.13560v1
  • PDF: https://arxiv.org/pdf/2608.13560v1
  • DOI: UNKNOWN
  • Registry group: academic_apis_and_preprints
  • Evidence role: canonical_academic_index
  • Language: en
  • Country / region: UNKNOWN

Plain-Language Brief

Researchers have developed a new framework called AutoDesign to improve the process of transforming multimodal sources into structured media outputs, such as converting academic papers into posters. The system uses a meta-harness optimizer to guide a code agent in recursively improving the harness based on feedback from rollouts. The researchers evaluated the effectiveness of the AutoDesign framework by comparing it to a commercial system named Claude Design and found that AutoDesign outperformed it on a task called PosterBench. The system also improved performance across various configurations, executed multiple tool calls and editing turns in less than an hour for under $3, and received the highest human preference among evaluated systems.

What the Document Says

  • The paper introduces AutoDesign, a framework to transform multimodal sources into structured media outputs
  • AutoDesign aligns with human design priors and uses a meta-harness optimizer to improve harness based on rollout feedback
  • The researchers focus on the academic paper-to-poster generation task and introduce PosterBench for evaluation
  • PosterBench contains a Main Track of 100 papers spanning five disciplines and a smaller subset for controlled evaluation
  • On the PosterBench Main Track, AutoDesign achieves the highest score of 78.32, surpassing Claude Design by 7.45 points
  • Integrating the learned DesignHarness improves performance across seven configurations, increasing the average PosterBench Score from 54.99 to 67.39 (+12.4%)
  • In a fully autonomous long-horizon loop, AutoDesign executes 253 tool calls and 11 editing turns within 40 minutes for under $3
  • The system achieves average conference-poster quality in human evaluation
  • A system-blind human study shows that AutoDesign receives the highest human preference among evaluated systems

Key Concepts Mentioned

  • AutoDesign
  • harness system
  • model-harness system
  • rollout feedback
  • academic paper-to-poster generation task
  • PosterBench
  • Claude Design
  • design priors
  • code agent
  • tool calls
  • editing turns

Claims Made by the Document

  • AutoDesign aligns with human design priors
  • The researchers introduce AutoDesign and PosterBench for evaluation
  • AutoDesign outperforms Claude Design on the PosterBench Main Track
  • Integrating the learned DesignHarness improves performance across configurations
  • AutoDesign can achieve average conference-poster quality in human evaluation

Evidence Used

  • The paper describes AutoDesign and its components
  • The text mentions the introduction of PosterBench for evaluation
  • The scores on the PosterBench Main Track are provided
  • The improvements across configurations due to integrating DesignHarness are shown
  • The human evaluation results are presented

Methods / Approach

  • Using a meta-harness optimizer to guide a code agent in recursively improving harness based on rollout feedback
  • Evaluating the framework using PosterBench and comparing it with Claude Design

Limitations Stated by the Authors

  • The paper does not discuss potential limitations or challenges of AutoDesign outside the context of the academic paper-to-poster generation task

Limitations Noted by the Agent

Agent interpretation — not from the source.

  • The agent is limited to the information provided in the text. Potential limitations or challenges not explicitly discussed by the authors may exist.
  • It's also important to note that the performance of AutoDesign may vary in different tasks or scenarios than the one described in the paper.

Open Questions

  • What are some potential limitations and challenges of AutoDesign outside the context of the academic paper-to-poster generation task?

Notes for Future Q&A

May include interpretation. Labeled where it does.

  • The authors do not discuss potential limitations of AutoDesign beyond the specific task of academic paper-to-poster generation. Additional research may be required to understand its limitations in other scenarios.

Raw Abstract / Excerpt

Transforming multimodal sources into condensed and structured media outputs can be fundamentally conceptualized as a long-horizon agentic process centered on a model-harness system. While an ideal harness system should align with human design priors and accumulate reusable experience through empirical exploration to drive recursive self-improvement, existing paradigms remain static and fall short of this capability. In this paper, we present AutoDesign, a framework that aligns with human design priors, where a meta-harness optimizer guides a code agent to recursively improve harness based on rollout feedback. To instantiate and evaluate this framework, we focus on the academic paper-to-poster generation task and introduce PosterBench, comprising a 100-paper Main Track spanning five disciplines and PosterBench-mini, a shared 10-paper subset for controlled evaluation. On the PosterBench Main Track, AutoDesign achieves the highest score of 78.32, surpassing the closed-source commercial system Claude Design by 7.45 points. Across seven controlled code-agent-model configurations, integrating the learned DesignHarness consistently improves performance, increasing the average PosterBench Score from 54.99 to 67.39 (+12.4%). In a fully autonomous long-horizon loop, it executes 253 tool calls and 11 editing turns within 40 minutes for under $3, reaching average conference-poster quality in human evaluation. A system-blind human study further demonstrates that AutoDesign achieves the highest human preference among evaluated systems.