TRIAGE: Role-Typed Credit Assignment for Agentic Reinforcement Learning

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Yuanda Xu, Zhengze Zhou, Hejian Sang, Xiaomin Li, Jiaxin Zhang, Xinchen Du, Zhipeng Wang, Alborz Geramifard
• Venue: arXiv: cs.LG
• Published: 2026-06-30
• Collected: 2026-07-01
• URL: http://arxiv.org/abs/2606.32017v1
• PDF: https://arxiv.org/pdf/2606.32017v1
• DOI: UNKNOWN

Plain-Language Brief

The paper proposes a new framework called TRIAGE for credit assignment in agentic reinforcement learning. This framework adds a semantic role axis to outcome credit, allowing for more accurate reward distribution for actions like searches, clicks, edits, etc. The framework is shown to improve success rates on several tasks and reduce environment-facing turns compared to the standard GRPO method.

What the Document Says

• The paper presents TRIAGE, a role-typed credit assignment framework for agentic reinforcement learning.
• TRIAGE uses a structured judge to classify each action segment as decisive progress, useful exploration, no-progress infrastructure, or regression.
• A fixed rule maps these labels to bounded segment-level process rewards based on the assigned roles.
• TRIAGE keeps verifier outcomes as the source of optimization direction while correcting the two main blind spots of outcome-only credit.
• The paper shows that role-conditioned credit is the optimal segment-level correction expressible from role labels alone.
• TRIAGE improves success rates over GRPO for two policy models and outperforms both a scalar judge-derived process reward and an outcome-supervised shared-backbone value baseline across ALFWorld, Search-QA, and WebShop.
• Ablations show that the gain comes from role typing rather than merely adding dense rewards.
• Reliable detection of regression inside successful trajectories is the dominant contributor, while exploration credit provides a consistent secondary gain.
• On completed ALFWorld and WebShop rollouts, TRIAGE reduces environment-facing turns by an additional 10.4% and 14.8% relative to GRPO.

Key Concepts Mentioned

• agentic reinforcement learning
• credit assignment
• TRIAGE
• role-typed credit assignment
• structured judge
• process rewards
• ALFWorld
• Search-QA
• WebShop

Claims Made by the Document

• TRIAGE corrects the two main blind spots of outcome-only credit.
• Role-conditioned credit is the optimal segment-level correction expressible from role labels alone.
• TRIAGE improves success rates over GRPO for two policy models.

Evidence Used

• The paper describes how TRIAGE works and its benefits.
• Results are presented showing improvements in success rates and reduced environment-facing turns.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Agentic reinforcement learning requires assigning credit to environment-facing actions such as searches, clicks, edits, navigation commands, and object interactions. Standard GRPO uses the final verifier outcome as a uniform advantage over all action tokens. This outcome signal is useful but structurally incomplete: it punishes useful exploration in failed rollouts and reinforces redundant or regressive actions in successful rollouts. We propose TRIAGE, a role-typed credit assignment framework that adds a semantic role axis to outcome credit. A structured judge classifies each segment as decisive progress, useful exploration, no-progress infrastructure, or regression, and a fixed role-conditioned rule maps these labels to bounded segment-level process rewards. This keeps verifier outcomes as the source of optimization direction while correcting the two main blind spots of outcome-only credit. We further show that role-conditioned credit is the optimal segment-level correction expressible from role labels alone -- a projection of the per-segment advantage residual onto the role variable -- so that the fixed role constants reduce advantage estimation error whenever the judge is reliable, and we connect this to lower-variance policy gradients. Across ALFWorld, Search-QA, and WebShop, TRIAGE improves success rates over GRPO for two policy models and outperforms both a scalar judge-derived process reward and an outcome-supervised shared-backbone value baseline. Ablations show that the gain comes from role typing rather than merely adding dense rewards: reliable detection of regression inside successful trajectories is the dominant contributor, while exploration credit provides a consistent secondary gain; on completed ALFWorld and WebShop rollouts, TRIAGE also reduces environment-facing turns by an additional $10.4%$ and $14.8%$ relative to GRPO.