SkillGate: Training In-Policy Skill Selection in Long-Horizon Agents

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Qingyao Li, Wenxiang Jiao, Shuai Shao, Kangning Zhang, Yuan Lu, Yi Guo, Weiwen Liu, Weinan Zhang, Yong Yu
• Venue: arXiv: cs.AI
• Published: 2026-08-19
• Collected: 2026-08-20
• URL: http://arxiv.org/abs/2608.18852v1
• PDF: https://arxiv.org/pdf/2608.18852v1
• DOI: UNKNOWN
• Registry group: academic_apis_and_preprints
• Evidence role: canonical_academic_index
• Language: en
• Country / region: UNKNOWN

Plain-Language Brief

Researchers present SkillGate, a method that aims to improve long-horizon agents' ability to select relevant skills during training. The current approach, outcome-rewarded Reinforcement Learning (RL), has limitations due to 'selector credit starvation', which results in the chosen skill receiving insignificant credit when trajectories fail, even though the choice itself is crucial. SkillGate addresses this issue by separating credits into two channels: outcome credit and action-local advantage. This separation ensures that only the correct skill receives positive credit. On five agentic benchmarks, SkillGate significantly improves trial success rates and reduces exposure to misleading skills compared to outcome reward alone.

What the Document Says

• Agent frameworks package procedural knowledge as skills.
• The decision of which skill to read is made by the policy itself during an episode.
• No existing signal trains this decision.
• Outcome-rewarded RL over the candidate slate cannot teach it due to a problem named 'selector credit starvation'.
• Under a broadcast, sequence-level advantage, the chosen skill carries a vanishing share of the loss.
• A correct choice is punished whenever the execution after it fails.
• Auditing a completed run's own training artifacts confirms these properties.
• SkillGate remedies this failure by partitioning token support into two disjoint credit channels: outcome credit and action-local advantage.
• Outcome credit reaches only execution tokens, while action-local advantage reaches exactly the skill-naming tokens.
• Action-local advantage is positive only when a trajectory's single read is the correct one.
• On five agentic benchmarks under a 16-candidate slate, SkillGate lifts a 9B policy from 40.8% to 53.2% trial success.
• SkillGate outperforms identical budget spent on outcome reward alone.
• SkillGate reduces exposure to misleading candidates by two thirds and reads fewer skills.

Key Concepts Mentioned

• Agent frameworks
• Procedural knowledge
• Skills
• Policy
• Selection decision
• Outcome-rewarded Reinforcement Learning (RL)
• Selector credit starvation
• Token support
• Credit channels
• Outcome credit
• Action-local advantage

Claims Made by the Document

• Outcome-rewarded RL cannot teach the selection decision due to selector credit starvation.
• SkillGate addresses the failure caused by selector credit starvation.

Evidence Used

• The text describes how SkillGate was developed to improve long-horizon agents' ability to select relevant skills during training.
• The text explains that under outcome-rewarded RL, the chosen skill carries a vanishing share of the loss due to selector credit starvation.

Methods / Approach

• SkillGate partitions token support into two disjoint credit channels: outcome credit and action-local advantage

Limitations Stated by the Authors

The authors do not discuss any potential limitations of their proposed method, SkillGate.

Limitations Noted by the Agent

Agent interpretation — not from the source.

• The agent may struggle with generalizing the learned skills to new tasks or environments outside the training data.
• The agent's performance may depend on the quality and relevance of the skills provided in the library.

Open Questions

• How does SkillGate perform compared to other methods for addressing selector credit starvation?
• What are the potential side effects or unintended consequences of separating credits into two channels as proposed by SkillGate?

Notes for Future Q&A

May include interpretation. Labeled where it does.

The text does not provide a comparison between SkillGate and other methods addressing selector credit starvation.

Raw Abstract / Excerpt

Agent frameworks increasingly package procedural knowledge as skills: instruction files an agent reads on demand, while public libraries now hold thousands of them. Which skill to read has thus become a decision the policy itself makes in the middle of an episode, yet no existing signal trains it. We show that the default remedy, outcome-rewarded RL over the candidate slate, cannot teach it, for a structural reason we identify and name selector credit starvation: under a broadcast, sequence-level advantage, the few tokens that name the chosen skill carry a vanishing share of the loss, and the credit they inherit is increasingly wrong-signed as trajectories lengthen. A correct choice is punished whenever the execution after it fails, even though the choice itself is among the most valuable decisions in the trajectory. Auditing a completed run's own training artifacts confirms all three properties, each worsening monotonically with horizon. SkillGate removes the failure by construction: it partitions the token support into two disjoint credit channels, outcome credit reaching only execution tokens, and a separate action-local advantage reaching exactly the skill-naming tokens, positive only when a trajectory's single read is the correct one. On five agentic benchmarks under a 16-candidate slate, SkillGate lifts a 9B policy from 40.8% to 53.2% trial success, well ahead of the identical budget spent on outcome reward alone, while cutting exposure to misleading candidates by two thirds and reading fewer skills.