EvoPolicyGym: Evaluating Autonomous Policy Evolution in Interactive Environments

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Zhilin Wang, Han Song, Runzhe Zhan, Jusen Du, Jiacheng Chen, Tianle Li, Qingyu Yin, Yulun Wu, Zhennan Shen, Tong Zhu, Yanshu Li, Guanjie Chen, Derek F. Wong, Yafu Li, Yu Cheng, Yang Yang
• Venue: arXiv: cs.AI
• Published: 2026-07-02
• Collected: 2026-07-03
• URL: http://arxiv.org/abs/2607.02440v1
• PDF: https://arxiv.org/pdf/2607.02440v1
• DOI: UNKNOWN

Plain-Language Brief

The paper introduces EvoPolicyGym, a new benchmark for evaluating how autonomous agents improve their executable policies through feedback in interactive environments. The authors use this benchmark to evaluate GPT-5.5 and demonstrate that strong policy evolution depends on discovering task-appropriate mechanisms and refining policies under bounded feedback.

What the Document Says

• The paper presents Autonomous Policy Evolution, a controlled evaluation setting for harness-model agents editing executable policy systems within a fixed interaction budget.
• EvoPolicyGym is introduced as a benchmark built from compact interactive reinforcement learning (RL) environments that evaluate agent performance in iteratively improving explored policies.
• The EvoPolicyGym suite is used to compare the performance of GPT-5.5 against other agents, with GPT-5.5 achieving the highest aggregate rank score and top-two performance on all 16 environments.
• EvoPolicyGym provides trajectory-level diagnostics that reveal how agents allocate budget and convert feedback into parametric tuning.
• The analyses show that strong autonomous policy evolution relies not only on isolated task wins, but also on discovering task-appropriate mechanisms and refining policies under bounded feedback.

Key Concepts Mentioned

• Autonomous agents
• Executable policies
• Feedback
• Policy Evolution
• Evaluation setting
• EvoPolicyGym
• Interactive RL environments
• GPT-5.5

Claims Made by the Document

• Existing evaluations often collapse the policy improvement process into a final score or confound it with open-ended software-engineering progress.
• GPT-5.5 achieves the strongest aggregate rank score and top-two performance on all 16 environments in the EvoPolicyGym suite.

Evidence Used

• The authors state that existing evaluations often lack control over the policy improvement process.
• Data from the EvoPolicyGym benchmark is provided to demonstrate GPT-5.5's performance.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Autonomous agents are increasingly expected to improve executable policies through feedback, yet existing evaluations often collapse this process into a final score or confound it with open-ended software-engineering progress. We introduce Autonomous Policy Evolution, a controlled evaluation setting in which a harness-model agent repeatedly edits an executable policy system under a fixed interaction budget. We instantiate this setting in EvoPolicyGym, a benchmark built from compact interactive RL environments that evaluates how agents iteratively improve explored policies. On the EvoPolicyGym suite, GPT-5.5 achieves the strongest aggregate rank score and top-two performance on all 16 environments. Beyond leaderboard results, EvoPolicyGym also provides trajectory-level diagnostics that distinguish how agents allocate budget, convert feedback into parametric tuning. These analyses show that strong autonomous policy evolution depends not only on isolated task wins, but on discovering task-appropriate mechanisms and refining policies under bounded feedback.