EnvACE: Internalizing Environment Dynamics via World Rehearsal for Agentic Reinforcement Learning

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Zishan Xu, Zhiyuan Yao, Yuxin Chen, Yifu Guo, Zhengxi Lu, Yuquan Lu, Jinyang Huang, Yan Xu, Yasheng Wang, Weinan Zhang, Xingshan Zeng, Weiwen Liu
• Venue: arXiv: cs.AI
• Published: 2026-08-06
• Collected: 2026-08-07
• URL: http://arxiv.org/abs/2608.06197v1
• PDF: https://arxiv.org/pdf/2608.06197v1
• DOI: UNKNOWN

Plain-Language Brief

Researchers have introduced EnvACE, a method for training large language model agents using world rehearsal instead of interactions with real or simulated environments. The method involves the policy alternating between acting and rehearsal, where it generates an action, plays the role of the environment to produce a response, and conditions subsequent decisions on the rehearsed response. All roles are jointly optimized end-to-end using task-success rewards. This approach allows the policy to internalize the relationship between actions and their environment responses, yielding an agent world model that directly supports decision making. The research shows that EnvACE achieves strong and transferable performance across several benchmarks, outperforming environment-scaling baselines in overall evaluation.

What the Document Says

• EnvACE is an agentic reinforcement learning method for training large language model agents.
• The method replaces external environment interaction during training with world rehearsal.
• The policy alternates between acting and rehearsal: it generates a tool call, then plays the role of the environment to produce the response induced by that action, and conditions subsequent decisions on the rehearsed response.
• Both roles are jointly optimized end-to-end using task-success rewards.
• Through world rehearsal, the policy internalizes the relationship between actions and their environment responses in its parameters.
• EnvACE achieves strong and transferable performance across BFCL-v4, tau^2-Bench, VitaBench, and FinMCP-Bench.
• EnvACE outperforms environment-scaling baselines in the overall evaluation.

Key Concepts Mentioned

• EnvACE
• agentic reinforcement learning
• world rehearsal
• task-success rewards
• policy
• environment responses

Claims Made by the Document

• EnvACE is a new path toward scaling LLM agent training beyond the constraints of external environments.

Evidence Used

• The findings establish world rehearsal as a new path toward scaling LLM agent training...

Methods / Approach

• Replacing external environment interaction with world rehearsal
• Policy alternates between acting and rehearsal
• Both roles are jointly optimized end-to-end using task-success rewards

Limitations Stated by the Authors

• UNKNOWN (Not stated in source)

Limitations Noted by the Agent

Agent interpretation — not from the source.

• The research only presents the introduction of EnvACE and its performance on several benchmarks. Further studies could investigate other aspects such as limitations, edge cases, or real-world applications.

Open Questions

• What are the potential limitations of EnvACE?
• How does EnvACE perform in real-world applications?

Notes for Future Q&A

May include interpretation. Labeled where it does.

• The paper focuses on introducing and testing EnvACE. For a comprehensive understanding, it would be beneficial to also consider potential limitations, edge cases, or real-world applications.

Raw Abstract / Excerpt

Training large language model agents for long-horizon tool use typically relies on interactions with real or synthesized executable environments, whose construction and verification are costly, or on external simulators that are difficult to ground. We introduce EnvACE, an agentic reinforcement learning method that replaces external environment interaction during training with world rehearsal. The policy alternates between acting and rehearsal: it first generates a tool call, then plays the role of the environment to produce the response induced by that action, and conditions subsequent decisions on the rehearsed response. Both roles are jointly optimized end-to-end using task-success rewards. Through world rehearsal, the policy internalizes the relationship between actions and their environment responses in its parameters, yielding an agent world model that directly supports decision making. Across BFCL-v4, tau^2-Bench, VitaBench, and FinMCP-Bench, EnvACE achieves strong and transferable performance, outperforming environment-scaling baselines in the overall evaluation. Controlled studies further show that world rehearsal consistently improves policy learning across model scales. At test time, the internalized world model enables private rehearsal before committed execution, yielding further gains under a moderate rehearsal budget without additional external interaction. Our findings establish world rehearsal as a new path toward scaling LLM agent training beyond the constraints of external environments. Our code is publicly available at https://github.com/Within-yao/EnvACE.