Paper Summary – CODESKILL

1. Introduction

SWE agents can produce rich trajectories.

To reuse the experience, one appoach is to use memory mechanism to store those information. Another way is to distill trajectories into skills.

One way to produce skill is that using LLM to analyze prior successes and failures, extract reusable lessons, workflows, or reasoning patterns, and update memory over time for future task solving.

However, there remains some problems:

• fixed prompts
• heuristic criteria
• the feedback of SWE are dense and heterogeneous(specific not general).

So the LLM-Generation method is difficult to distinguish resuable procedural knowledge from task-specific or accidental details.

The article proposed an LLM-based framework, CODESKILL, optimized to generate reusable skills from coding-agent trajectories and maintain a skill bank for augmenting future softwareengineering tasks.

Given past trajectories and an existing skill bank, CODESKILL extracts reusable procedural knowledge as skills, evolves existing skills based on new or failed experience, and maintains the skill bank by adding useful candidates, merging redundant ones, or dropping unhelpful skills.

To train the CODESKILL:

• SFT with trajectories and teacher-generated skills
• GRPO

GRPO reward:

• Verifiable task feedback
• Rubric-based judgments
  • skill quality
  • behavior-skill alignment

The authors validate CODESKILL by instantiating it with Qwen3.5-4B and evaluating it on EnvBench, SWEBench Verified, and Terminal-Bench 2.

Contributions:

• CODESKILL, an LLM-based framework that analyzes coding-agent trajectories across diverse tasks, extracts reusable skills, and maintains an evolving skill bank for software-engineering tasks.
• Reformulate skill management as a learnable management policy, and train CODESKILL with reinforcement learning. The reward combines sparse verifiable feedback with dense rubric-based judgments, balancing executable outcomes with informative supervision when task success is sparse.

2. CODESKILL

2.1 Problem Formalization

• A frozen downstream coding policy π
• SWE task x ∈ 𝒳
• Rollout trajectory τ = (o₁, a₁, o₂, a₂…o_T, a_T, y) where o_t is an observation, a_t is an action such as shell command.

Goal is to learn from such trajectories without updating π.

CODESKILL learns a policy M_θ for managing a skill bank ℬ = {s_i}_i = 1^N where each skill s_i constrains reusable procedural instructions that provide actionable guidance.

Given trajectory evidence τ and relevant skill-bank context 𝒞 ⊆ ℬ.

M_θ outputs an operation u = (a, z), where a ∈ 𝒜 denotes the operation type, such as generation, evolution, or maintenance. And z denotes the operation content, such as a generated skill or a maintenance decision.

Applying u to produces an updated skill bank ℬ^′ = Update (ℬ, u).

The objective is to optimize M_θ so that the updated skill bank improves the downstream performance of the frozen policy π when used as prior knowledge.

$$ \begin{aligned} &\max_{\theta}\ \mathbb{E}_{\tau,\,x'}\left[R_{\mathrm{task}}\left(\pi(x' \mid \mathcal{B}')\right)\right], \\ &\quad\ u = M_{\theta}(\tau, \mathcal{C}), \quad \mathcal{B}' = \operatorname{Update}(\mathcal{B}, u). \end{aligned} $$ R_task denotes the downstream task performance on future task x^′.

2.2 Skill Management Loop

Figure 1 provides an overview of the skill-management framework of CODESKILL. In this section, we describe how CODESKILL organizes the skill bank and manages it through three major components, including skill extraction, skill evolution, and skill-bank maintenance, together with their corresponding operations.

3.2.1 Multi-granularity Skill Bank

Skills can be classified into 2 granularities:

• Task-level (high): Skills capture high-level strategies for a task or a family of related tasks, such as how to inspect the repository, localize the issue, or validate a fix. They are often distilled from trajectories that solve related queries.
• Event-driven (low): They specify how the agent should react when the corresponding event is triggered, and can transfer across tasks because similar execution events recur across software-engineering problems.

3.2.2 Skill Bank Construction

An evidence is a combination of trajectory, result and context.

Skill Extraction:

Skill Evolution:

3.2.3 Skill Maintenance

The skill library should keep compact.

Each newly extracted or evolved candidate skill is further passed to a maintenance stage.

Similar skills are retrieved from the current bank and provided to M_θ together with the candidate. Based on this context, M_θ outputs 3 maintenance operations: add, merge and drop.

• Add: The add operation inserts the candidate as a new skill when it provides useful knowledge not covered by existing skills.
• Merge: The merge operation combines the candidate with an existing skill when they overlap but contain complementary guidance.
• Drop: The drop operation rejects candidates that are redundant, weakly grounded, overly specific, or unlikely to transfer.