1. Introduction

SWE agents can produce rich trajectories.

To reuse the experience, one appoach is to use memory mechanism to store those information. Another way is to distill trajectories into skills.

One way to produce skill is that using LLM to analyze prior successes and failures, extract reusable lessons, workflows, or reasoning patterns, and update memory over time for future task solving.

However, there remains some problems:

  • fixed prompts
  • heuristic criteria
  • the feedback of SWE are dense and heterogeneous(specific not general).

So the LLM-Generation method is difficult to distinguish resuable procedural knowledge from task-specific or accidental details.

The article proposed an LLM-based framework, CODESKILL, optimized to generate reusable skills from coding-agent trajectories and maintain a skill bank for augmenting future softwareengineering tasks.

Given past trajectories and an existing skill bank, CODESKILL extracts reusable procedural knowledge as skills, evolves existing skills based on new or failed experience, and maintains the skill bank by adding useful candidates, merging redundant ones, or dropping unhelpful skills.

To train the CODESKILL:

  • SFT with trajectories and teacher-generated skills
  • GRPO

GRPO reward:

  • Verifiable task feedback
  • Rubric-based judgments
    • skill quality
    • behavior-skill alignment

The authors validate CODESKILL by instantiating it with Qwen3.5-4B and evaluating it on EnvBench, SWEBench Verified, and Terminal-Bench 2.

Contributions:

  • CODESKILL, an LLM-based framework that analyzes coding-agent trajectories across diverse tasks, extracts reusable skills, and maintains an evolving skill bank for software-engineering tasks.
  • Reformulate skill management as a learnable management policy, and train CODESKILL with reinforcement learning. The reward combines sparse verifiable feedback with dense rubric-based judgments, balancing executable outcomes with informative supervision when task success is sparse.

2. CODESKILL

2.1 Problem Formalization

  • A frozen downstream coding policy \(\pi\)
  • SWE task \(x \in \mathcal{X}\)
  • Rollout trajectory \(\tau = \left(o_1, a_1, o_2, a_2 \dots o_T, a_T, y\right)\) where \(o_t\) is an observation, \(a_t\) is an action such as shell command.

Goal is to learn from such trajectories without updating \(\pi\).

CODESKILL learns a policy \({M_\theta}\) for managing a skill bank \(\mathcal{B} = \{s_i\}_{i=1}^{N}\) where each skill \(s_i\) constrains reusable procedural instructions that provide actionable guidance.

Given trajectory evidence \(\tau\) and relevant skill-bank context \(\mathcal{C \subseteq B}\).

[!NOTE]

Here the main LLM (Qwen3.5-35B-A3B) is frozen;

There’s an additional LLM \(M_\theta\) (Qwen3.5-4B) to maintain the skill bank. This work is to train the \(M_\theta\).

\(M_\theta\) outputs an operation \(u = (a,z)\), where \(a \in \mathcal{A}\) denotes the operation type, such as generation, evolution, or maintenance. And \(z\) denotes the operation content, such as a generated skill or a maintenance decision.

Applying \(u\) to produces an updated skill bank \(\mathcal{B}' = \operatorname{Update}(\mathcal{B}, u)\).

The objective is to optimize \(M_\theta\) so that the updated skill bank improves the downstream performance of the frozen policy \(\pi\) when used as prior knowledge.

\[ \begin{aligned} &\max_{\theta}\ \mathbb{E}_{\tau,\,x'}\left[R_{\mathrm{task}}\left(\pi(x' \mid \mathcal{B}')\right)\right], \\ &\quad\ u = M_{\theta}(\tau, \mathcal{C}), \quad \mathcal{B}' = \operatorname{Update}(\mathcal{B}, u). \end{aligned} \] \(R_{task}\) denotes the downstream task performance on future task \(x'\).

2.2 Skill Management Loop

Figure 1 provides an overview of the skill-management framework of CODESKILL. In this section, we describe how CODESKILL organizes the skill bank and manages it through three major components, including skill extraction, skill evolution, and skill-bank maintenance, together with their corresponding operations.

3.2.1 Multi-granularity Skill Bank

Skills can be classified into 2 granularities:

  • Task-level (high): Skills capture high-level strategies for a task or a family of related tasks, such as how to inspect the repository, localize the issue, or validate a fix. They are often distilled from trajectories that solve related queries.
  • Event-driven (low): They specify how the agent should react when the corresponding event is triggered, and can transfer across tasks because similar execution events recur across software-engineering problems.

3.2.2 Skill Bank Construction

An evidence is a combination of trajectory, result and context.

Skill Extraction:

Skill Evolution:

3.2.3 Skill Maintenance

The skill library should keep compact.

Each newly extracted or evolved candidate skill is further passed to a maintenance stage.

Similar skills are retrieved from the current bank and provided to \(M_\theta\) together with the candidate. Based on this context, \(M_\theta\) outputs 3 maintenance operations: add, merge and drop.

  • Add: The add operation inserts the candidate as a new skill when it provides useful knowledge not covered by existing skills.
  • Merge: The merge operation combines the candidate with an existing skill when they overlap but contain complementary guidance.
  • Drop: The drop operation rejects candidates that are redundant, weakly grounded, overly specific, or unlikely to transfer.

3.2.4 Skill Usage (Skill Match)

For downstream task solving, relevant skills are retrieved from the updated skill bank and inserted into the prompt of the frozen coding policy as prior knowledge.

Retrieval is based on dense semantic similarity, matching task-level skills to the task goal and event-driven skills to the agent’s current reasoning and observation through their trigger conditions.

The original task prompt and policy parameters remain unchanged.

2.3 Model Training

The authors first warm up \(M_\theta\) on teacher-generated supervised data for skill extraction, evolution, and maintenance, and then optimize it with reinforcement learning using feedback from both rubric-based LLM-as-judge rewards and a frozen downstream coding policy.

2.3.1 SFT

2.3.1.1 Brief

The SFT’s goal is to make \(M_\theta\) learn basic skill bank operations.

As we mentioned earlier, the \(M_\theta\) outputs three main operations.

  • add
  • merge
  • drop

Operation \(u\) can be expressed as: \[ u = (a,z) \]

  • \(a\) denotes operation type
  • \(z\) denotes operation content.

The \(u\) is goal of SFT.

The teacher produces structured targets \(u^*\) for skill extraction, evolution, and maintenance: \[ u^* = (a^*, z^*) \] Each SFT example is written as: \[ (q, u^*), \ q= \left(\tau, \tilde s, \mathcal C \right) \]

  • \(\tau\) denotes trajectory evidence
  • \(\tilde s\) denotes an optional candidate or existing skill
  • \(\mathcal C\) denotes retrieved similar skills

SFT loss: \[ \mathcal{L}_{\mathrm{SFT}} = -\mathbb{E}_{(q,u^*)}\log M_\theta(u^* \mid q). \]

2.3.1.2 Trajectories Sources

The initial SWE-style trajectory pool is downloaded from publicly available mini-SWE-agent rollouts on SWE-smith, where Qwen330B is used as the coding agent.

The authors additionally collect EnvBench trajectories using a ReAct-style agent.

Since different sources use different logging formats, the authors normalize all trajectories into a unified representation containing:

  • task context
  • reasoning steps
  • action steps
  • observation steps
  • final outcome information

2.3.1.3 Teacher Data Construction

The SFT is to teach student 3 things: skill extraction, skill evolution and skill bank maintenance.

  • For task-level and event-driven extraction:

    the teacher reads normalized trajectory evidence and outputs either a reusable skill or a skip decision.

  • For skill evolution:

    The authors synthesize training instances by retrieving an existing skill whose applicability condition or rules match a new trajectory, especially a failed or partially successful one.

    The paired trajectory is selected to expose missing conditions, better procedures, or failure modes not fully covered by the existing skill, and the teacher is asked to revise the skill or skip when no meaningful update is supported.

    This avoids collecting an additional round of skill-conditioned rollouts.

  • For Skill Maintenance:

    For skill-bank maintenance, each candidate skill is paired with retrieved similar skills from the bank, and the teacher outputs add, merge, or drop.

2.3.1.4 SFT Training Setting

2.3.3 Training Settings for CODESKILL

2.2.2 GRPO

2.2.2.1 Hybrid Reward Design

2.2.2.1.1 Skill Quality Reward by LLM-as-Judge

To train \(M_\theta\) with RL, the authors first introduce a rubric-based LLM-as-judge quality reward \(R_Q \in [0,1]\) to score each generated operation and its content.

For skills, the rubric evaluates grounding, reusability, specificity, format, and actionability, with action-specific criteria.

3.2 Prompts

2.2.2.1.2 Execution Reward

However, quality reward alone is not execution-grounded, since a skill may look plausible but remain unused or ineffective.

The authors then define an execution reward by measuring the skill’s incremental effect on the frozen coding policy \(π\).

  • Select a valid skill \(s_u\) which only can be generated by operation generate, add and drop. By reverse retrieval, \(x_u \sim \operatorname{TopK}\!\left(s_u,\ D_{\mathrm{task}}\right)\), where the skill content retrieves task goals or baseline trajectories and \(x_u\) is a task sampled from the top \(K\) results.

  • Calculate no-skill baseline of the selected task: \[ b_{\pi}(x_u) = \frac{1}{n}\sum_{i=1}^{n} V\!\left(\tau^{0}_{\pi,i}\right), \qquad \tau^{0}_{\pi,i} \sim \pi(\cdot \mid x_u) \]

  • The authors then run a skill-conditioned rollout: \[ \tau^{u}_{\pi} \sim \pi(\cdot \mid x_u,\ s_u) \]

  • The execution improvement: \[ R_E(u;\, x_u, \pi) = V\!\left(\tau^{u}_{\pi}\right) - b_{\pi}(x_u) \] Here \(\tau^{0}_{\pi,i}\) is the \(i\)-th no-skill rollout, \(\tau^{u}_{\pi}\) is the rollout conditioned on the skill produced by \[u\], and \(V(\tau) \in [0,1]\) is the task verifier score.

  • Execution improvement alone has an attribution problem. The coding agent may solve the task for reasons unrelated to the skill, or fail despite following a useful skill because of other errors in the long-horizon rollout.

  • The authors then use an alignment factor \(R_A\!\left(u;\, \tau^{u}_{\pi}\right) \in [0, 1]\) to judge whether the rollout actually matches the skill trigger condition and follows its workflow or constraints. The final reward for skill-output operations is \[ R(u; q) = \lambda\, R_Q(u; q) + R_A\!\left(u; \tau^{u}_{\pi}\right)\, R_E(u;\, x_u, \pi) \]

    • \(R_Q\): Quality reward by LLM-as-Judge
    • \(\lambda\): 0.25 according to training settings.
    • \(R_A\): alignment factor
    • \(R_E\): execution reward

2.2.2.2 Policy Optimization Loop

  • For each prompt \(q\), we sample a group of \(G\) operations \(\{ u_j \}_{j=1}^G\) from the old policy \(M_{\theta_{\text{old}}}\).

  • Advantages: \[ \hat{A}_j = \frac{R_j - \mu_R}{\sigma_R + \epsilon_A}, \qquad j = 1, \dots, G \]

    • \(R_j\): reward of the \(j\)-th operation.
    • \(\mu_R\): average of all Rewards of group \(G\).
    • \(\sigma_R\): standard deviation, normalization
    • \(\epsilon_A\): avoid division by zero

  • Update \(M_\theta\) by advantage:

    …

2.3.3 Training Settings for CODESKILL

3. Skill Examples & Prompts

3.1 Skill Examples

3.1.1 Task-Level Skill Example

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
**Title**
Diagnose Maven build failures via environment, config, and dependency resolution.

**Granularity**
general

**When to Apply**
When a Java/Maven project fails to build and the cause may involve toolchain or wrapper configuration, repository settings, missing or stale dependencies, or build-phase ordering rather than source code errors.

**Instructions**
• Start by identifying the project type and build instructions from repository metadata such as the main build file and README before making changes.  
• Run the build with the project’s Maven wrapper first, then capture both early and later output to distinguish dependency download progress from the actual failure.  
• Use the first concrete failure message to distinguish network or repository issues, missing artifacts, and compilation errors before changing code.  
• If the build fails in the enforcer or toolchain phase, use verbose output to identify the required Java and Maven versions before changing anything.  
• Check the active Java and Maven toolchain versions in the shell and align them with the project’s documented requirements when needed.  
• When a dependency cannot be resolved, check whether the missing artifact is generated later in the build, and verify whether the build phase order or plugin lifecycle is appropriate.  
• Inspect local Maven settings and repository configuration before changing code; misconfigured mirrors or missing settings files can mask the real build problem.  
• After adjusting toolchain, wrapper settings, repository configuration, or dependency versions, rerun the build and re-check the logs to confirm whether the failure mode has changed.

**Provenance**
This skill is an example from the maintained skill bank and was produced by a merge operation.

3.1.2 Event-Driven Skill Example

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
**Title**
Handle missing Maven test-jar dependencies.

**Granularity**
event-driven

**When to Apply**
When a Maven build fails with a dependency resolution error for a test-jar or similar scoped artifact that has not yet been produced or installed.

**Instructions**
• Inspect the failing module’s POM to confirm the dependency type and scope, for example a test-jar with compile scope.  
• Check the producing module’s POM to see whether it already defines an execution for the required artifact via a plugin configuration.  
• If the test jar is only generated when tests run, either run the tests or explicitly invoke the Maven test-jar goal to create the test artifact before rebuilding.  
• If the artifact is missing, add or configure the appropriate plugin execution in the producing module’s POM to generate the required artifact, and place the new plugin configuration inside the correct parent element so the POM remains well formed.  
• Do not simply retry the same Maven install command with tests skipped; skipping tests often prevents the test jar from being created.  
• After generating or configuring the artifact, run a full install/build command so the artifact is generated and installed into the local repository; a compile-only run may not install the needed artifact. 
• Verify that the artifact is actually present in the local Maven repository before retrying the original failing command. 
• After the artifact is installed, rerun the original failing command to confirm that the dependency resolution error is resolved.

**Provenance**
This skill is an example from the maintained skill bank and was produced by a merge operation.

3.2 Prompts

3.2.1 Skill Operation Prompts

3.2.1.1 Task-Level Skill Extraction Prompt

3.2.1.2 Event-Driven Skill Extraction Prompt

3.2.1.3 Skill Evolution Prompt

3.2.1.4 Skill-Bank Maintenance Prompt

3.2.2 Judge Prompts

3.2.2.1 Task-Level Skill Quality Judge Prompt

3.2.2.2 Event-Driven Skill Quality Judge

3.2.2.3 Skill Evolution Quality Judge Skill

3.2.2.4 Skill-Bank Maintenance Merge Judge Prompt

3.2.2.5 Behavior Alignment Judge Prompt

4. Conclusion

This work introduce an LLM to manage Skill Moduel of Agent.

The Skill Moduel, also known as Skill Bank, Skill Librabry or Skill Directory, is to store skills for Agent.

The tradition Agent driven by one LLM only read / write / add / delete the skills by the LLM.

This work uses an additional LLM to manage the Skill Bank, the this LLM is a small-parameter LLM (4B). This LLM is trained by SFT and GRPO to manage the Skill Bank.

The main operation the small-parameter LLM can do is mainly:

  • Skill Extraction
  • Skill Evolution
  • Skill Maintenance