EvoPolicyGym: Evaluating Autonomous Policy Evolution in Interactive Environments

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Zhilin Wang, Han Song, Runzhe Zhan, Jusen Du, Jiacheng Chen, Tianle Li, Qingyu Yin, Yulun Wu, Zhennan Shen, Tong Zhu, Yanshu Li, Guanjie Chen, Derek F. Wong, Yafu Li, Yu Cheng, Yang Yang
• Venue: arXiv: cs.AI
• Published: 2026-07-02
• Collected: 2026-07-03
• URL: http://arxiv.org/abs/2607.02440v1
• PDF: https://arxiv.org/pdf/2607.02440v1
• DOI: UNKNOWN

Plain-Language Brief

A new evaluation setting called Autonomous Policy Evolution (APE) is introduced in a benchmark named EvoPolicyGym, which is used to test how autonomous agents iteratively improve their policy systems within a fixed interaction budget. The authors compare the performance of various agents on 16 interactive reinforcement learning environments and find that GPT-5.5 performs the best overall and shows that strong APE depends on discovering task-appropriate mechanisms and refining policies under bounded feedback.

What the Document Says

• A new controlled evaluation setting called Autonomous Policy Evolution (APE) is introduced.
• The setting is instantiated in a benchmark named EvoPolicyGym, which evaluates how agents iteratively improve explored policies using compact interactive reinforcement learning environments.
• The performance of various autonomous agents on the EvoPolicyGym suite is compared.
• GPT-5.5 achieves the strongest aggregate rank score and top-two performance on all 16 environments.
• EvoPolicyGym provides trajectory-level diagnostics to distinguish how agents allocate budget and convert feedback into parametric tuning.
• The analyses show that strong APE depends on discovering task-appropriate mechanisms and refining policies under bounded feedback.

Key Concepts Mentioned

• Autonomous Policy Evolution
• APE
• EvoPolicyGym
• interactive reinforcement learning environments
• GPT-5.5
• trajectory-level diagnostics

Claims Made by the Document

• GPT-5.5 performs the best overall on the EvoPolicyGym suite.

Evidence Used

• On the EvoPolicyGym suite, GPT-5.5 achieves the strongest aggregate rank score and top-two performance on all 16 environments.

Methods / Approach

• Comparison of the performance of various autonomous agents using a new evaluation setting called Autonomous Policy Evolution (APE) in a benchmark named EvoPolicyGym

Limitations Stated by the Authors

The authors do not mention any limitations in their paper.

Limitations Noted by the Agent

Agent interpretation — not from the source.

It is unknown if there are any limitations to the study that were not mentioned by the authors.

Open Questions

• What other autonomous agents were tested on EvoPolicyGym and how did they perform compared to GPT-5.5?
• How does the performance of GPT-5.5 compare to human agents in these interactive reinforcement learning environments?

Notes for Future Q&A

May include interpretation. Labeled where it does.

• The authors do not provide information about other autonomous agents tested in EvoPolicyGym.
• The authors do not compare the performance of GPT-5.5 to human agents.

Raw Abstract / Excerpt

Autonomous agents are increasingly expected to improve executable policies through feedback, yet existing evaluations often collapse this process into a final score or confound it with open-ended software-engineering progress. We introduce Autonomous Policy Evolution, a controlled evaluation setting in which a harness-model agent repeatedly edits an executable policy system under a fixed interaction budget. We instantiate this setting in EvoPolicyGym, a benchmark built from compact interactive RL environments that evaluates how agents iteratively improve explored policies. On the EvoPolicyGym suite, GPT-5.5 achieves the strongest aggregate rank score and top-two performance on all 16 environments. Beyond leaderboard results, EvoPolicyGym also provides trajectory-level diagnostics that distinguish how agents allocate budget, convert feedback into parametric tuning. These analyses show that strong autonomous policy evolution depends not only on isolated task wins, but on discovering task-appropriate mechanisms and refining policies under bounded feedback.