Joint Learning of Experiential Rules and Policies for Large Language Model Agents

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Shicheng Ye, Chao Yu
• Venue: arXiv: cs.AI
• Published: 2026-06-25
• Collected: 2026-06-26
• URL: http://arxiv.org/abs/2606.27136v1
• PDF: https://arxiv.org/pdf/2606.27136v1
• DOI: UNKNOWN

Plain-Language Brief

A research paper proposes a method called Joint Learning of Experiential Rules and Policies (JERP) to improve the effectiveness of Large Language Model Agents in multi-step interactive environments. The method updates both a long-term experiential-rule pool and the policy from the same interaction trajectories. It retrieves task-relevant rules at decision time, conditions the agent with them along with the interaction history, and optimizes the policy and revises the rule pool after each episode.

What the Document Says

• The research focuses on Large Language Model Agents (LLM) in multi-step interactive environments.
• Existing methods either keep experiential rules outside the model or update the model parameters using trajectories and feedback, but they have limitations.
• A new method called JERP is presented which updates a long-term experiential-rule pool and the policy from the same interaction trajectories.
• At decision time, JERP retrieves task-relevant rules and conditions the agent on them along with the interaction history.
• After each episode, it uses the collected trajectories to optimize the policy and revise the rule pool by comparing current rollouts with reference successful trajectories.

Key Concepts Mentioned

• Large Language Model Agents
• Experiential Rules
• Policies
• Interactive Environments
• Joint Learning
• AlfWorld
• WebShop

Claims Made by the Document

• JERP yields consistent gains in decision performance for complex interactive tasks.
• The method keeps the rule pool aligned with the evolving policy.

Evidence Used

• Experiments on AlfWorld and WebShop show that JERP yields consistent gains in decision performance

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

For LLM agents in multi-step interactive environments, a key challenge is to make effective use of accumulated interaction experience. Existing work has typically separated two uses of such experience: keeping it outside the model as natural-language rules for later prompting, or using trajectories and feedback to update the model parameters. The former is easy to interpret but can fall out of sync with the evolving policy; the latter improves the policy more broadly but provides only limited correction for local mistakes in sparse-reward settings. We present Joint Learning of Experiential Rules and Policies for LLM Agents (JERP), which updates a long-term experiential-rule pool and the policy from the same interaction trajectories. At decision time, JERP retrieves task-relevant rules and conditions the agent on them together with the interaction history. After each episode, it uses the collected trajectories both to optimize the policy and to revise the rule pool by comparing current rollouts with reference successful trajectories. This coupling keeps the rule pool aligned with the evolving policy while allowing stable and effective behaviors to be gradually absorbed into the model itself. Experiments on AlfWorld and WebShop show that JERP yields consistent gains in decision performance for complex interactive tasks.