SkillForge: Self-Distilling Agents for Project-Specific Issue Resolution

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Silin Chen, Han Li, Xiaodong Gu, Yuling Shi, Haibing Guan
• Venue: arXiv: cs.SE
• Published: 2026-08-19
• Collected: 2026-08-20
• URL: http://arxiv.org/abs/2608.18933v1
• PDF: https://arxiv.org/pdf/2608.18933v1
• DOI: UNKNOWN
• Registry group: academic_apis_and_preprints
• Evidence role: canonical_academic_index
• Language: en
• Country / region: UNKNOWN

Plain-Language Brief

SkillForge is a new framework that helps large language models (LLMs) improve their ability to resolve specific software issues by actively acquiring project-specific knowledge. Instead of waiting for real problems, SkillForge creates synthetic issues based on the re-implementation of core functionalities in the repository. By resolving these synthetic issues, it distills and stores reusable project-specific knowledge that can aid in future issue resolution.

What the Document Says

• SkillForge is a self-distilling framework for improving large language model (LLM) performance in software issue resolution. It focuses on project-specific issues.
• It proactively acquires project-specific knowledge by synthesizing and resolving issues based on the re-implementation of test-covered core functionalities from the repository.
• The distilled, reusable project-specific knowledge is stored as entity-grounded skills and associated with relevant repository entities.
• Experiments using both open-source and closed-source models show that SkillForge improves issue resolution performance over strong baselines.

Key Concepts Mentioned

• SkillForge
• large language model (LLM)
• software issue resolution
• project-specific knowledge
• synthetic issues

Claims Made by the Document

• SkillForge improves large language model performance in software issue resolution.
• Proactively acquiring project-specific knowledge before solving real issues substantially improves downstream software issue resolution.

Evidence Used

• Extensive experiments using both open-source and closed-source models show that SkillForge consistently improves issue resolution performance over strong baselines.

Methods / Approach

• Self-distillation framework
• Synthesizing and resolving synthetic issues based on re-implementing test-covered core functionalities from the repository.
• Storing distilled project-specific knowledge as entity-grounded skills and associating them with relevant repository entities.

Limitations Stated by the Authors

Not stated in source

Limitations Noted by the Agent

Agent interpretation — not from the source.

The paper does not discuss potential limitations of SkillForge. For instance, the effectiveness could be influenced by the quality and complexity of the test-covered core functionalities in the repository.

Open Questions

• What are the strong baselines against which SkillForge is being compared?
• How does SkillForge handle repositories with limited or no test coverage?

Notes for Future Q&A

May include interpretation. Labeled where it does.

The paper presents a new framework for improving the performance of large language models in software issue resolution. It emphasizes the proactive acquisition of project-specific knowledge and the use of synthetic issues for this purpose.

Raw Abstract / Excerpt

Large language model (LLM) based agents have demonstrated remarkable proficiency in automated software issue resolution, yet they often struggle to resolve issues in a specific repository because they lack project-specific knowledge. Existing self-evolving approaches acquire such knowledge from repository history or online repair trajectories, but they either depend on available historical issue-resolution signals or incur substantial per-issue test-time exploration cost. In this paper, we propose SkillForge, a self-distillation framework that proactively acquires project-specific knowledge from the repository itself. Instead of waiting for real issues to expose project-specific knowledge gaps, SkillForge synthesizes project-specific issues by re-implementing test-covered core functionalities of the repository. By resolving these synthetic issues, SkillForge distills reusable project-specific knowledge into entity-grounded skills and associates them with relevant repository entities for future issue resolution. Extensive experiments using both open-source and closed-source models show that SkillForge consistently improves issue resolution performance over strong baselines. These results demonstrate that proactively acquiring project-specific knowledge before solving real issues substantially improves downstream software issue resolution.