Beyond Trial-and-Error: Agentic Optimization for Image-to-Video Adherence

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Aman Tyagi, Hemanth Boinpally, Jonathan Chen, Douglas Gebert, Steven Hickson
• Venue: arXiv: cs.CV
• Published: 2026-08-12
• Collected: 2026-08-13
• URL: http://arxiv.org/abs/2608.12290v1
• PDF: https://arxiv.org/pdf/2608.12290v1
• DOI: UNKNOWN
• Registry group: academic_apis_and_preprints
• Evidence role: canonical_academic_index
• Language: en
• Country / region: UNKNOWN

Plain-Language Brief

The researchers introduce a new framework called 'Agentic Self-Improvement' to address challenges in professional workflows related to modern black-box Image-to-Video (I2V) models. The framework optimizes the video synthesis process through a closed-loop, goal-directed optimization approach that involves refining input prompts and using Bayesian optimization for efficient parameter search guided by quality metrics.

What the Document Says

• The text discusses a new framework called 'Agentic Self-Improvement' to enhance the predictability and control of modern black-box Image-to-Video (I2V) models.
• The framework navigates the generation parameter space using an iterative prompt optimization loop that refines input prompts through a multimodal Large Language Model.
• Two automated evaluations are implemented: Davidsonian Scene Graph (DSG) queries for semantic adherence and Common Mistake Questions (CMQ) for artifact detection.
• In the second stage, Bayesian optimization is used to co-optimize stochastic seeds and CFG scales.
• The search is guided by a suite of quality metrics, including the novel Video-Text Adherence (VTA) score derived from DSG and CMQ evaluations.
• The framework significantly outperforms unguided search methods in human preference studies, with win rates up to 69%.

Key Concepts Mentioned

• Agentic Self-Improvement
• Image-to-Video models
• prompt optimization loop
• multimodal Large Language Model
• Davidsonian Scene Graph (DSG) queries
• Common Mistake Questions (CMQ)
• Bayesian optimization
• Videos
• quality metrics
• Video-Text Adherence (VTA) score

Claims Made by the Document

• The new framework significantly outperforms unguided search methods.

Evidence Used

• In human preference studies, videos generated via our agentic approach were strongly preferred over baseline outputs, achieving win rates up to 69%.

Methods / Approach

• Iterative prompt optimization loop
• Multimodal Large Language Model
• Davidsonian Scene Graph (DSG) queries
• Common Mistake Questions (CMQ)
• Bayesian optimization

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Modern black-box Image-to-Video (I2V) models offer powerful capabilities in automated content creation, yet their lack of fine-grained control and reliability presents significant challenges in professional workflows. Their inherent stochasticity causes minor variations in textual prompts or hyperparameters to yield drastically different outputs often necessitating inefficient, brute-force trial-and-error processes. To address these limitations, we introduce the ``Agentic Self-Improvement' framework, which reframes video synthesis into a closed-loop, goal-directed optimization. Our framework systematically navigates the generation parameter space using a novel two-stage approach. In the first stage, an iterative prompt optimization loop uses a multimodal Large Language Model (mLLM) to refine the input prompt. This refinement implements two automated evaluations: Davidsonian Scene Graph (DSG) queries ensure semantic adherence, and Common Mistake Questions (CMQ) for artifact detection. At the second stage, we use Bayesian optimization to efficiently co-optimize stochastic seeds and CFG scales. This search is guided by a suite of quality metrics, including the novel Video-Text Adherence (VTA) score derived from the DSG and CMQ evaluations. Our framework significantly outperforms unguided search methods: in human preference studies, videos generated via our agentic approach were strongly preferred over baseline outputs, achieving win rates up to 69%. This work provides a practical and extensible methodology for enhancing the predictability and control of state-of-the-art video generation models, moving the field beyond speculative curiosities toward reliable, production-ready tools.