TRIAGE: Role-Typed Credit Assignment for Agentic Reinforcement Learning

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Yuanda Xu, Zhengze Zhou, Hejian Sang, Xiaomin Li, Jiaxin Zhang, Xinchen Du, Zhipeng Wang, Alborz Geramifard
• Venue: arXiv: cs.LG
• Published: 2026-06-30
• Collected: 2026-07-01
• URL: http://arxiv.org/abs/2606.32017v1
• PDF: https://arxiv.org/pdf/2606.32017v1
• DOI: UNKNOWN

Plain-Language Brief

The paper proposes a new framework called TRIAGE for credit assignment in agentic reinforcement learning. This framework adds a semantic role axis to outcome credit, aiming to correct the issues of standard GRPO where the final verifier outcome is used as a uniform advantage over all action tokens. TRIAGE uses a structured judge to classify each segment of an action into categories such as decisive progress, useful exploration, no-progress infrastructure, or regression. A fixed role-conditioned rule then maps these labels to bounded segment-level process rewards. The framework improves success rates on ALFWorld, Search-QA, and WebShop compared to GRPO for two policy models.

What the Document Says

• The paper presents TRIAGE, a credit assignment framework for agentic reinforcement learning.
• TRIAGE adds a semantic role axis to outcome credit.
• A structured judge classifies each segment of an action into four categories: decisive progress, useful exploration, no-progress infrastructure, or regression.
• A fixed role-conditioned rule maps these labels to bounded segment-level process rewards.
• TRIAGE improves success rates on ALFWorld, Search-QA, and WebShop compared to GRPO for two policy models.
• TRIAGE outperforms a scalar judge-derived process reward and an outcome-supervised shared-backbone value baseline.

Key Concepts Mentioned

• Agentic reinforcement learning
• Credit assignment
• TRIAGE framework
• Outcome credit
• Semantic role axis
• Structured judge
• Decisive progress
• Useful exploration
• No-progress infrastructure
• Regression

Claims Made by the Document

• TRIAGE corrects the two main blind spots of outcome-only credit.
• Role-conditioned credit is the optimal segment-level correction expressible from role labels alone.

Evidence Used

• The paper provides experiments on ALFWorld, Search-QA, and WebShop that show improved success rates with TRIAGE compared to GRPO.
• The paper proves that the fixed role constants reduce advantage estimation error whenever the judge is reliable.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Agentic reinforcement learning requires assigning credit to environment-facing actions such as searches, clicks, edits, navigation commands, and object interactions. Standard GRPO uses the final verifier outcome as a uniform advantage over all action tokens. This outcome signal is useful but structurally incomplete: it punishes useful exploration in failed rollouts and reinforces redundant or regressive actions in successful rollouts. We propose TRIAGE, a role-typed credit assignment framework that adds a semantic role axis to outcome credit. A structured judge classifies each segment as decisive progress, useful exploration, no-progress infrastructure, or regression, and a fixed role-conditioned rule maps these labels to bounded segment-level process rewards. This keeps verifier outcomes as the source of optimization direction while correcting the two main blind spots of outcome-only credit. We further show that role-conditioned credit is the optimal segment-level correction expressible from role labels alone -- a projection of the per-segment advantage residual onto the role variable -- so that the fixed role constants reduce advantage estimation error whenever the judge is reliable, and we connect this to lower-variance policy gradients. Across ALFWorld, Search-QA, and WebShop, TRIAGE improves success rates over GRPO for two policy models and outperforms both a scalar judge-derived process reward and an outcome-supervised shared-backbone value baseline. Ablations show that the gain comes from role typing rather than merely adding dense rewards: reliable detection of regression inside successful trajectories is the dominant contributor, while exploration credit provides a consistent secondary gain; on completed ALFWorld and WebShop rollouts, TRIAGE also reduces environment-facing turns by an additional $10.4%$ and $14.8%$ relative to GRPO.