Beyond Trial-and-Error: Agentic Optimization for Image-to-Video Adherence

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Aman Tyagi, Hemanth Boinpally, Jonathan Chen, Douglas Gebert, Steven Hickson
• Venue: arXiv: cs.CV
• Published: 2026-08-12
• Collected: 2026-08-13
• URL: http://arxiv.org/abs/2608.12290v1
• PDF: https://arxiv.org/pdf/2608.12290v1
• DOI: UNKNOWN
• Registry group: academic_apis_and_preprints
• Evidence role: canonical_academic_index
• Language: en
• Country / region: UNKNOWN

Plain-Language Brief

Researchers present the 'Agentic Self-Improvement' framework to address challenges in professional workflows with modern Image-to-Video (I2V) models. The framework uses a two-stage approach: an iterative prompt optimization loop with multimodal Large Language Model, automated evaluations for semantic adherence and artifact detection, Bayesian optimization for co-optimizing stochastic seeds and CFG scales, and quality metrics including the Video-Text Adherence (VTA) score. The method outperforms unguided search methods significantly in human preference studies, achieving win rates up to 69%.

What the Document Says

• Modern I2V models have challenges due to lack of fine-grained control and reliability.
• A novel framework named 'Agentic Self-Improvement' is introduced for video synthesis using closed-loop, goal-directed optimization.
• The first stage uses an iterative prompt optimization loop with a multimodal Large Language Model, automated evaluations for semantic adherence (DSG queries) and artifact detection (Common Mistake Questions).
• In the second stage, Bayesian optimization co-optimizes stochastic seeds and CFG scales.
• The search is guided by a suite of quality metrics including Video-Text Adherence score derived from DSG and CMQ evaluations.
• The framework outperforms unguided search methods in human preference studies with win rates up to 69%.

Key Concepts Mentioned

• Image-to-Video models
• Agentic Self-Improvement framework
• multimodal Large Language Model
• Davidsonian Scene Graph (DSG) queries
• Common Mistake Questions (CMQ)
• Bayesian optimization
• Video-Text Adherence (VTA) score

Claims Made by the Document

• Modern I2V models have challenges due to lack of fine-grained control and reliability.
• The 'Agentic Self-Improvement' framework outperforms unguided search methods significantly.

Evidence Used

• Lack of fine-grained control and reliability mentioned in the text.
• Win rates up to 69% mentioned in the text.

Methods / Approach

• Iterative prompt optimization loop
• Automated evaluations
• Bayesian optimization

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

The study only presents results from human preference studies, more quantitative metrics could be useful for further analysis.

Open Questions

• What is the specific I2V model used in the research?
• How was the unguided search method defined?

Notes for Future Q&A

May include interpretation. Labeled where it does.

The text does not provide details about the specific I2V model used or the exact definition of the unguided search method.

Raw Abstract / Excerpt

Modern black-box Image-to-Video (I2V) models offer powerful capabilities in automated content creation, yet their lack of fine-grained control and reliability presents significant challenges in professional workflows. Their inherent stochasticity causes minor variations in textual prompts or hyperparameters to yield drastically different outputs often necessitating inefficient, brute-force trial-and-error processes. To address these limitations, we introduce the ``Agentic Self-Improvement' framework, which reframes video synthesis into a closed-loop, goal-directed optimization. Our framework systematically navigates the generation parameter space using a novel two-stage approach. In the first stage, an iterative prompt optimization loop uses a multimodal Large Language Model (mLLM) to refine the input prompt. This refinement implements two automated evaluations: Davidsonian Scene Graph (DSG) queries ensure semantic adherence, and Common Mistake Questions (CMQ) for artifact detection. At the second stage, we use Bayesian optimization to efficiently co-optimize stochastic seeds and CFG scales. This search is guided by a suite of quality metrics, including the novel Video-Text Adherence (VTA) score derived from the DSG and CMQ evaluations. Our framework significantly outperforms unguided search methods: in human preference studies, videos generated via our agentic approach were strongly preferred over baseline outputs, achieving win rates up to 69%. This work provides a practical and extensible methodology for enhancing the predictability and control of state-of-the-art video generation models, moving the field beyond speculative curiosities toward reliable, production-ready tools.