EvoPolicyGym: Evaluating Autonomous Policy Evolution in Interactive Environments

Source

Source name: arXiv
Source kind: paper
Authors / organization: Zhilin Wang, Han Song, Runzhe Zhan, Jusen Du, Jiacheng Chen, Tianle Li, Qingyu Yin, Yulun Wu, Zhennan Shen, Tong Zhu, Yanshu Li, Guanjie Chen, Derek F. Wong, Yafu Li, Yu Cheng, Yang Yang
Venue: arXiv: cs.AI
Published: 2026-07-02
Collected: 2026-07-03
URL: http://arxiv.org/abs/2607.02440v1
PDF: https://arxiv.org/pdf/2607.02440v1
DOI: UNKNOWN

Plain-Language Brief

The research introduces EvoPolicyGym, a controlled evaluation setting for autonomous agents to improve executable policies through feedback. The setting is designed to separate this process from a final score or software-engineering progress. The study presents results from using GPT-5.5 on the EvoPolicyGym suite, which shows that GPT-5.5 achieves the highest aggregate rank score and top-two performance in all 16 environments.

What the Document Says

The research introduces a controlled evaluation setting called Autonomous Policy Evolution.
The setting is for harness-model agents to repeatedly edit an executable policy system within a fixed interaction budget.
EvoPolicyGym, a benchmark built from compact interactive reinforcement learning (RL) environments, is used to evaluate how agents iteratively improve explored policies.
Results are presented from using GPT-5.5 on the EvoPolicyGym suite.
GPT-5.5 achieves the highest aggregate rank score and top-two performance in all 16 environments.
EvoPolicyGym also provides trajectory-level diagnostics that show how agents allocate budget and convert feedback into parametric tuning.

Key Concepts Mentioned

Autonomous Policy Evolution
EvoPolicyGym
harness-model agent
executable policy system
interaction budget
compact interactive RL environments
GPT-5.5
aggregate rank score

Claims Made by the Document

Autonomous Policy Evolution is a controlled evaluation setting.
GPT-5.5 achieves the highest aggregate rank score and top-two performance on all 16 environments in EvoPolicyGym.

Evidence Used

The text states that GPT-5.5 achieved the strongest aggregate rank score and top-two performance on all 16 environments.

Methods / Approach

Controlled evaluation setting for autonomous policy evolution
EvoPolicyGym benchmark with compact interactive RL environments

Limitations Stated by the Authors

Not stated in source

Limitations Noted by the Agent

Agent interpretation — not from the source.

The research does not provide information on the limitations of the study or the methodology.

Open Questions

What are the specifics of the GPT-5.5 model used in the study?
Are there other agents that perform better than GPT-5.5 in certain environments within EvoPolicyGym?

Notes for Future Q&A

May include interpretation. Labeled where it does.

The research does not provide detailed information about the GPT-5.5 model used, making it an open question for further investigation.

Raw Abstract / Excerpt

Autonomous agents are increasingly expected to improve executable policies through feedback, yet existing evaluations often collapse this process into a final score or confound it with open-ended software-engineering progress. We introduce Autonomous Policy Evolution, a controlled evaluation setting in which a harness-model agent repeatedly edits an executable policy system under a fixed interaction budget. We instantiate this setting in EvoPolicyGym, a benchmark built from compact interactive RL environments that evaluates how agents iteratively improve explored policies. On the EvoPolicyGym suite, GPT-5.5 achieves the strongest aggregate rank score and top-two performance on all 16 environments. Beyond leaderboard results, EvoPolicyGym also provides trajectory-level diagnostics that distinguish how agents allocate budget, convert feedback into parametric tuning. These analyses show that strong autonomous policy evolution depends not only on isolated task wins, but on discovering task-appropriate mechanisms and refining policies under bounded feedback.