1. Introduction

There have been two ways of skill evolving:

  • Sequential Skill Editing: Update/Edit Skill after each incoming trace.
  • ReasoningBank-style Retrieval Memories: Distill experience from an agent’s past executions into transferable memory items, store them in an external memory bank, and at inference time retrieve the semantically most relevant items and inject them into the agent’s context for reuse.

The thought of Trace2Skill likes Human Experts: inspect broad traces, abstract recurring patterns, and write compact procedures that are reusable across cases.

2. Trace2Skill

There three stages of Trace2Skill:

  • Collect Trajectories
  • Propose Trajectory-Level Patches in Parallel
  • Consolidate Patches into One Portable Skill

2.1 Skill & Problem Formalization

A skill \(S=\left(M, \mathcal R \right)\) nowadays mainly consists of two parts:

  • The root SKILL.md file, denoted as \(M\)
  • The auxiliary files including references, scripts, or assets denoted as \(\mathcal R\).

The root document \(M\) stores broadly applicable procedural knowledge, while auxiliary files \(\mathcal R\) provide deterministic tools or lower-frequency details.

Let \(\pi_\theta\) be a fixed LLM agent using skill \(S\).

Let \(\mathcal P \left(S;\pi_{\theta}, \mathcal D\right)\) denote the pass rate of that agent on task set \(\mathcal D\).

An evolving set \(\mathcal D_{evolve}\), and a disjoint test set \(\mathcal D_{test}\), skill evolution constructs a new skill from evolving-set trajectories without updating \(\theta\): \[ S^* = \mathcal{E}(S_0, D_{evolve}; \pi_\theta), \qquad P(S^*; \pi_\theta, D_{test}) > P(S_0; \pi_\theta, D_{test}). \]

[!NOTE]

The evolving set \(\mathcal D_{evolve}\) is to evolve the skill; the \(\mathcal D_{evolve}\) itself does not change.

The work evaluates two initializations for \(S_0\):

  • human-expert skill
  • LLM-generated draft skill

2.2 Stage 1: Trajectory Generation

This stage uses a Re-Act-Style harness. For each evolving-set task, the fixed agent runs with \(S_0\) and produces a trajectory \(\tau_i\) containing the query, reasoning/tool-use history, final output, and a binary correctness outcome.

The resulting corpus \(\mathcal T\) is split into:

  • Failures $T^- $
  • Successes $T^+ $

2.3 Stage 2: Parallel Patch Proposal

A group of analyst sub-agents independently proposes skill patches from individual trajectories.

Failures are sent to an error analyst \(\mathcal A^-\), successes to a success analyst \(\mathcal A^+\).

The analyst read \(\mathcal S_0\) and propose a patch for it, leading to a patch pool. \[ \mathcal {P = P^- \cup P^+ } \] The analyst roles are intentionally asymmetric.

  • \(\mathcal A^+\) uses a single-pass workflow to identify reusable behavior patterns from successful trajectories.
    1. Read trajectory
    2. Abstract reusable pattern
    3. Read \(S_0\), write a skill patch
  • \(\mathcal A^-\) uses a ReAct-style loop that can inspect traces and artifacts, compare outputs against ground truth, and validate candidate fixes before proposing a patch.
    1. Read trajectory + artifacts + ground-truth value
    2. Suppose a reason for the failure
    3. Write a minimal repair fix to verify the failure reason. If the reason is not correct , go back to step 2.
    4. If the reason confirmed, write a skill patch.

In short, for failure trajectory, this stage needs analyst to find out the reason.

Here’s a example of the skill patch:

2.4 Stage 3: Patch Consolidation

In stage 2, the work produces many patches storing in a patch pool.

In stage 3, the work needs to consolidate the patches into one coherent update patch \(p^*\) and apply it to \(S_0\).

Patches are merged hierarchically for \(L = \lceil \log_{B_{\text{merge}}} |P| \rceil\) levels; at each level \(\ell\), up to \(B_{\text{merge}}\) patches are synthesized into one patch: \[ p^{(\ell+1)} = \mathcal{M}\left(\pi_\theta,\ S_0,\ \{p^{(\ell)}_1, \ldots, p^{(\ell)}_{B_{\text{merge}}}\}\right), \qquad \ell = 0, \ldots, L-1 \] where \(\mathcal{M}\) deduplicates, resolves conflicts, and preserves non-overlapping insights.

\(\pi_\theta\) itself serves as trajectory generator, analyst, and merge operator, so no external evolution/teacher model is required.

The final \(p^∗\) is translated into diff-style edits and applied with deterministic guardrails:

  • reject edits to missing files
  • withhold line-range conflicts
  • validate the updated skill format

2.4.1 Patch Consolidation

Now we have:

  • Patch Pool: \(P\) = \(P^+\cup P^-\)
  • Preliminary Skill: \(S=\left(M, \mathcal R \right)\)

Now we need to consolidate the patches from patch pool into the \(S_0\).

There are too many patches to be consolidated. If all patches are consolidated at the same time, there may have many conflicts. So the authors consolidate the patches in many times. \[ L = \lceil \log_{B_{\text{merge}}} |P| \rceil \]

  • \(B_\text{merge}\): merge batch size, how many patches can be merged in one round.
  • \(L\): merge level.
  • \(P\): size of patch pool; how many patches need to be merged.

e.g. 323 patches in patch pool, \(B_\text{merge}\) = 5.

  • Level 1:

    $323 %  5 = 64 3 $; total 65 merges;

    In 64 times, \(\mathcal M\) (Merge Operator, the LLM itself) merges 5 patches;

    And \(\mathcal M\) merge last 3 patches in last round;

    Totally generate 65 new patches.

  • Level 2:

    $65 %  5 = 13 $

    Merge 65 patches into 13 next level patches.

  • Level 3:

    $13 %  5 = 2 3 $;

    Totally 3 next level patches.

  • Level 4:

    Merge 3 patches into final patch \(p^*\).

What happens in a single merge operation: \[ p^{(\ell+1)} = \mathcal{M}\left(\pi_\theta,\ S_0,\ \{p^{(\ell)}_1, \ldots, p^{(\ell)}_{B_{\text{merge}}}\}\right), \qquad \ell = 0, \ldots, L-1 \]

  • \(\mathcal M\): merge operation
  • \(\pi_\theta\): LLM
  • \(S_0\): preliminary Skill
  • \(p\): patch

\(\mathcal M\) is instructed to prefer prevalent patterns and discard idiosyncratic edits, producing a compact skill that replaces \(S_0\) and is used directly at inference without retrieval.

The final product of Patch Consolidation is a json-format patch likes:

2.4.2 Patch Application

The merge operator \(\mathcal M\) outputs JSON instead of a diff because diffs are made of line numbers, which LLMs handle poorly; JSON lets the model specify what to change using semantic anchors (target_section) while the program computes where.

It also makes the patch mechanically validatable (missing files, overlapping line ranges, format) and leaves room for a reasoning trace and changelog_entries that a diff cannot hold.

But in the end, we need a diff-format file to apply.

So we need to transfer the JSON file into diff:

3. Conclusion

Panorama:

Trajectory Analyst: