Panorama of Agent

Posted on Edited on In Views:
This article describes the panorama of Agent area. It is suggested to review this article frequently.

1. What is Agent & Harness?

First, the Large Language Model(LLM) is very mature now. Its essence is a prediction model:

  • You input a string of token, and it outputs a string of token.

The LLM nowadays can answer our questions but it can not finish a job independently. For example, you can ask LLM how to create a new folder on PC and it can give you an answer. However, the LLM cannot create the folder directly because the only thing it can do is output tokens.

Now we have LLM to describe the workflow of a job. But we need to finish the workflow ourselves. ==Agent== is the software that can resolve problems independently.

==Agent== is usually composed two parts: ==LLM== and ==Harness==.

Agent = LLM + Harness

LLM is responsible for thinking, and Harness is responsible for executing operations. e.g. LLM thought it needs to read a file, then the Harness part will read the file and return the result to LLM.

In Agent, LLM is more likely a role of “Brain”, and the Harness is the “Limb”.

2. Agent’s Pursuit

Agent is designed to solve problem independently. Everything we do now is to improve Agent’s ability of solving problems.

We hope Agent can solve particular problems in daily life and complex problems in specific area like software engineering.

3. Agent Area Panorama

Agent is composed of LLM and Harness.

So it is obviously that we should enhance LLM’s performance in Agent area. And optimize the Harness part. Then we need benchmarks to judge the Agent’s performance.

The main research area can be divided into 4 main parts:

  • LLM Tuning
  • Harness Structure Optimization
  • Benchmark relative work
  • Multi-agent relative work

3.1 LLM Tuning

In Agent area, the most popular post-training methods are:

  • Imitation Learning (SFT)
  • Online RL
  • Offline RL
  • Preference Learning (DPO)

3.1.1 Imitation Learning (SFT)

ℒSFT = −∑tlog Pθ(tokent ∣ token < t) #### 3.1.1.1 Trajectory SFT

Core Process:

1
2
3
4
5
6
7
Expert Agent Trajectory
        ↓
Teacher Forcing
        ↓
Cross Entropy
        ↓
LLM

Trajectory (ReAct) :

1
2
3
4
5
6
7
task
→ thought
→ action
→ observation
→ action
→ ...
→ final answer

Relative papers:

  • SWE-Gym (ICML 2025), Rejection Sampling / Self-Improvement
  • SWE-Lego (arXiv:2601.01426, 2026)

3.1.1.2 Critical-Step SFT

A trajectory includes many steps, but not all steps are necessary. There’s one way only to SFT the critical steps in trajectories.

  • ATLaS — (Findings ACL 2025), only train critical steps

3.1.1.3 Synthetic Trajectory

One problem of SFT is that high-quality training data is few, so there’s a research direction is to compose Synthetic Trajectory.

  • SWE-smith, Synthetic Trajectory

3.1.1.4 Distillation

Distillation is to extract ==reasoning trajectories== form different teachers and distill into a student model.

1
2
3
4
5
6
7
8
9
10
11
12
13
Teacher A
  │
  ├── Tool-based reasoning
  │
Teacher B
  │
  └── Text reasoning
       │
       ▼
     Distill
       │
       ▼
Student Agent
  • Agentic-R1 (EMNLP 2025)

3.1.2 Online RL

3.1.2.1 Reward Design

The reward is a criterion to judge how good a trajectory is.

There are many ways to calculate reward:

1
2
3
4
5
6
7
            Reward
              │
  ┌───────────┼────────────┐
  │           │            │
Human      Verifier    Process Reward
  │           │            │
 RLHF        RLVR       PRM/PR

RLVR (Reinforcement Learning Verifiable Reward) is an important type, because many engineering problems are verifiable.

  • SWE-RL (NeurIPS 2025)
  • WebAgent-R1 (EMNLP 2025)

Process Reward Model(PRM) is another research roadmap; There are many steps in a trajectory. The traditional reward is trajectory-level, which means that LLM cannot distinguish if one certain step in a trajectory is important.

1
2
3
4
5
6
7
8
9
Step 1
Step 2
Step 3
...
Step 50
 ↓
Success
 ↓
Reward = 1
  • LLM cannot know if step 3 is really important.
1
2
3
4
5
Outcome Reward
       ↓
Process Reward
       ↓
Step-level Credit Assignment
  • Rewarding Progress (ICLR 2025)
  • PURE (NeurIPS 2025)
  • ReasonFlux-PRM (NeurIPS 2025)

3.1.2.2 RL Optimization Algorithm

After calculating the reward of a trajectory, we need to update the weights of LLM.

There are three ways to update weights, but the PMD is not commonly used.

1
2
3
4
5
6
7
8
9
10
11
12
         Reward
           │
           ▼
   Policy Optimization
           │
 ┌─────────┼─────────┐
 │         │         │
PPO       GRPO      PMD
 │         │         │
 └─────────┼─────────┘
           ▼
      Update LLM
3.1.2.2.1 GRPO

GRPO is the most popular RL optimization algorithm because it doesn’t need a value/critic model which is necessary for PPO.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
Question x
   │
   ├── rollout 1 → R1
   ├── rollout 2 → R2
   ├── rollout 3 → R3
   ├── ...
   └── rollout G → RG
                 │
                 ▼
      Group Relative Advantage
                 │
                 ▼
                GRPO
                 │
                 ▼
                LLM
  • SWE-RL
3.1.2.2.2 PPO

The core thought of PPO (Proximal Policy Optimization) is:

  • Need to make LLM more inclined to generate high-reward responses, while ensuring that a single update does not deviate too far from the original model.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
Question x
    │
    ▼
Policy πold
    │
    ▼
rollout → trajectory
    │
    ├── Reward R
    │
    └── States / Actions
              │
              ▼
        Value / Critic V(s)
              │
              ▼
        Advantage A
              │
              ▼
        PPO Objective
              │
        ┌─────┴─────┐
        │           │
   Probability    Clip
      Ratio
        │           │
        └─────┬─────┘
              ▼
             Loss
              │
              ▼
       Update Policy πθ

3.1.3 Offline RL

  • OREO (ACL 2025): Offline Reinforcement Learning for LLM Multi-step Reasoning
  • Reward-Weighted Fine-Tuning (NeurIPS 2025): Offline RL by Reward-Weighted Fine-Tuning for Conversation Optimization

3.1.4 Preference Learning (DPO)

DPO (Direct Preference Optimization) is prevail in LLM post-training area, but in Agent area it’s not as popular as SFT and Online RL.

SFT makes LLM realize what is right answer while Preference Learning just tells LLM: A is better than B.

1
2
3
4
Task

Trajectory A -> good
Trajectory B -> bad

Train:

1
2
πθ(A|x) ↑
πθ(B|x) ↓

Basic DPO process:

1
2
3
4
5
6
7
Prompt
 ├── Chosen
 └── Rejected
       ↓
      DPO
       ↓
      LLM

The traditional DPO compare different trajectories. Nowadays the tendency is to research which certain step makes trajectory better.

1
2
3
4
5
6
7
Trajectory-level
       ↓
Step-level
       ↓
Action-group level
       ↓
Critical-step preference
  • HPL (ICLR 2026): Solving the Granularity Mismatch: Hierarchical Preference Learning for Long-Horizon LLM Agents
  • Online DPO (ICLR 2025)
  • SDPO (ACL 2025)

3.2 Harness Structure

Harness is the part that help LLM finish job. It can be divided into 7 main parts nowadays.

Part Abstruct Description
Orchestration
Loop / Planning / ReAct / Reflection / Verification		 Control How should the agent decide what to do?
Context & Memory
Context Management / Retrieval / Compression / Memory		 State What should the agent remember?
Tools & Actions
Tool Calling / Discovery / Selection / Code / Browser / Computer		 Action What can the agent do?
ACI & Environment
State / Action Space / Observation / Error / Feedback		 Interface How should the world be exposed to the agent?
Skills & Composition
Skills / Workflow / Subagents / Agent-as-a-Tool		 Capability How can reusable expertise be packaged?
Protocol & Interoperability
MCP / A2A / Agent API / Session Protocol		 Interoperability How do agents/tools/services communicate?
Runtime & Safety
Sandbox / Execution / Permission / Guardrail / Reliability / Observability / Cost		 Runtime & Governance Where and how are actions executed?
How do we keep agents safe, reliable, observable and economical?		 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
Agent
│
├── LLM
│
└── Harness
    │
    ├── 1. Orchestration
    │      Loop / Planning / ReAct / Reflection / Verification
    │
    ├── 2. Context & Memory
    │      Context Management / Retrieval / Compression / Memory
    │
    ├── 3. Tools & Actions
    │      Tool Calling / Discovery / Selection / Code / Browser / Computer
    │
    ├── 4. ACI & Environment
    │      State / Action Space / Observation / Error / Feedback
    │
    ├── 5. Skills & Composition
    │      Skills / Workflow / Subagents / Agent-as-a-Tool
    │
    ├── 6. Protocol & Interoperability
    │      MCP / A2A / Agent API / Session Protocol
    │
    └── 7. Runtime & Safety
           Sandbox / Execution / Permission / Guardrail /
           Reliability / Observability / Cost

3.2.1 Orchestration / Control

Q: How should the Agent decide what to do?

1
2
3
4
5
6
Orchestration
├── Loop
├── Planning
├── ReAct
├── Reflection
└── Verification

Earlier Agent(Classic ReAct):

1
LLM → Tool → LLM → Tool

Nowadays Agent:

1
Plan → Act → Verify → Reflect → Re-plan → Act

From simple ReAct Loop to a Loop with planning / verification / recovery.

  • PlanGEN (EMNLP 2025): This work combines planning and iterative verification

  • ReflAct (EMNLP 2025): The traditional ReAct sturcure will accumulate little errors which will make agent drift from correct path.(ALFWorld + 27.7% than ReAct)

  • 1
    2
    3
    4
    5
    6
    7
    Current State
          ↕
    Goal State
          ↓
    Reflection
          ↓
    Action

  • Metagent-P (ACL 2025)

3.2.2 Context & Memory / State

Q: What should the Agent know at the current moment, and what past information should be retained ?

1
2
3
4
5
6
7
Orchestration
    ↓
“What should I do?”

Context & Memory
    ↓
“What should I know?”

The Context & Memory is not just ‘Chat Log’. It store the Agent State.

1
2
3
4
5
6
7
8
9
Current Context
+
Working Memory
+
Long-term Memory
+
Retrieved Experience
+
Compressed History

The ==tradition memory system==:

1
2
3
Memory
=
Vector DB + Embedding + Retrieval

Nowadays:

1
2
3
4
5
6
Agent decides:
    store?
    update?
    delete?
    retrieve?
    summarize?

Memory Management will be considered into Agent Policy.

  • Memory OS of AI Agent, (EMNLP 2025)

    This article design the Memory System like operating system hierarchy.

    1
    2
    3
    4
    5
    Short-term
        ↓
    Mid-term
        ↓
    Long-term

  • Coarse-to-Fine Grounded Memory, (EMNLP 2025)

    This article use memory to replan. Memory is used to store useful things.

    1
    2
    3
    4
    5
    6
    7
    Past Experience
          ↓
    Grounded Memory
          ↓
    Current Situation
          ↓
    Planning

  • Agentic Memory, ACL 2026

  • Memory-R1，ACL 2026

One problem:

How Memory Management Impacts LLM Agents, ACL 2026:

The agent will clearly exhibit “experience-following”: after retrieving historical experience from similar tasks, it is easy to reproduce similar behaviors. So if there exists wrong memory, the Agent are inclined to repeat the wrong operations.

3.2.3 Tools & Actions / Action

What can Agent do ?

The tradition actions system is simply call tools like:

1
2
3
4
5
6
7
8
Search
Code
Database
Browser
Filesystem
Shell
Computer
API

The new tendency of actions area is:

1
2
3
4
5
Tool Discovery
Tool Selection
Tool Composition
Tool Creation
Computer Use

  • LLM Agents Making Agent Tools，ACL 2025

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    Paper + GitHub Code
            ↓
    Agent
            ↓
    Install dependencies
            ↓
    Generate tool
            ↓
    Execute
            ↓
    Debug / Self-correct

    This paper makes Agent generate create tools itself.

  • Adaptive Tool Use，ACL 2025

    More tools do not necessarily mean a better Agent. If one tool frequently returns errors, the Agent should realize that.

  • ToolScope，ACL 2026

    In large tool ecosystem, there exists a problem:

    1
    2
    3
    4
    5
    6
    7
    Too Many Tools 
    + 
    Repeated Tools 
    + 
    Tedious Schema
    +
    Trouble Choosing

    Solution:

    1
    2
    3
    4
    5
    Tool Merging
    +
    Tool Retrieval
    +
    Context-aware Filtering

3.2.4 ACI & Environment / Interface

This part is to solve the problem: What form should the external world take for the agent, and how should the agent act upon it?

The tradition agent takes a form like API Agent:

1
search(query)

The newly Computer-Agent (Imitate Human’s Operation)：

1
2
3
4
5
6
7
8
9
Screenshot
  ↓
LLM
  ↓
click(x,y)
  ↓
Screenshot
  ↓
...

Advantage: Developer need no more to design tools for each job.

  • OS Agents Survey，ACL 2025:

    The paper defines OS Agent that interact with:

    1
    2
    3
    4
    5
    GUI
    CLI
    Web
    Mobile
    Desktop

  • OSWorld-Human, ICML 2025:

    To finish a certain task, the Agent takes more steps/operations than human. This article

    1
    2
    3
    4
    5
    Agent Benchmark
         ↓
    Can you finish?
         +
    Can you finish efficiently?

3.2.5 Skills & Composition / Capability

How can we extract complex, repeatable capabilities from a specific agent and turn them into reusable, composable capabilities?

1
2
Tool = One Action
Skill = A method of finishing one certain kind job.
1
2
Tools → Atomic Capability
Skills → procedural capability

Tendency: Skill are changing to software artifact from prompt.

The traditional Skill is a text:

1
2
3
4
5
To finish ... task, you should:
1. ...
2. ...
3. ...
4. ...

It’s a stable text describing a fixed workflow.

  • Agent Skills，Anthropic 2025

    Skill is no more a huge text file. It’s new structure:

    1
    2
    3
    4
    Skill/
    ├── instructions
    ├── scripts
    └── resources

    e.g:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    pdf-to-excel/
    ├── SKILL.md
    ├── scripts/
    │   ├── extract_table.py
    │   ├── clean_data.py
    │   └── validate.py
    └── resources/
        ├── excel_template.xlsx
        └── examples/

    There’s a small Skill metadata like tool definitions will be read by Agent when system starts:

    1
    2
    3
    4
    5
    6
    7
    Available Skill:

    Name:
    pdf-to-excel

    Description:
    Extract structured tables from PDFs and produce validated Excel files.

    When handling a certain task:

    1
    2
    3
    4
    5
    Task
    ↓
    Skill Retrieval
    ↓
    Load Relative Skill, e.g. pdf-to-excel/SKILL.md

    In Skill.md, there may exists content like:

    1
    2
    3
    4
    5
    6
    7
    ...

    Run:

    python scripts/extract_table.py report.pdf

    ...

    In fact the scripts are tools for skills.

    GitHub - anthropics/skills: Public repository for Agent Skills · GitHub

  • SkillWeaver, arXiv:2504.07079

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    Web Task
       ↓
    Agent Exploration
       ↓
    Successful Trajectory
       ↓
    Skill Discovery
       ↓
    Skill Synthesis
       ↓
    Skill Practice / Debugging
       ↓
    Reusable Skill API
       ↓
    Skill Library
       ↓
    Future Tasks

  • SkillRL (arXiv:2602.08234)

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    Experience
         ↓
    Skill Distillation
         ↓
    Hierarchical SkillBank
         ↓
    Retrieval
         ↓
    RL Training
         ↓
    Policy Improvement
         ↓
    New Experience
         ↓
    SkillBank Evolution

  • SkillCraft, (arXiv:2603.00718)

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    Task A
     ↓
    Learn Skill
     ↓
    Task B
     ↓
    Reuse Skill
     ↓
    Task C
     ↓
    Reuse / Compose Skill

  • Generative Skill Composition for LLM Agents, (arXiv:2606.32025)

    How to compose skills.

  • Agent Skills for Large Language Models: Architecture, Acquisition, Security, and the Path Forward, (arXiv:2602.12430)

3.2.6 Protocol & Interoperability / Interoperability

Solve the problems of communication between Agents, Tools, Data Source and Services.

  • MCP (Model Context Protocol):

    1
    2
    3
    Agent
     ↕
    Tools / Data / Services

  • A2A (Agent to Agent):

    Different Agents should have the capability to work together.

    1
    Agent A ↔ Agent B

3.2.7 Runtime & Safety —— Runtime & Governance

Make the Agent actually run in a controlled, secure, observable, and recoverable environment.

Traditional safety check:

1
2
3
4
5
Prompt
 ↓
Output
 ↓
Safety Check

Agent workflow:

1
2
3
4
5
6
7
8
9
Prompt
 ↓
Agent
 ↓
Tool Call
 ↓
External Data
 ↓
Side Effect

So we should check safety of not only prompt and output but also Action, Tool Call, State Transition and Permission.

  • Task Shield，ACL 2025

    Does this Action is really relative to User’s task ?

  • IPIGuard，EMNLP 2025

    Create a Tool Dependency Graph, and limit the tool call by the dependency graph.

3.2.8 Conclusion

Field 2025 Tendency in 2026
Orchestration ReAct、Planning、Reflection Adaptive / verified / long-horizon control
Context & Memory RAG、hierarchical memory、context compression Agentic memory、learned memory management
Tools & Actions Tool calling、tool selection Tool discovery、tool management、tool creation、computer use
ACI & Environment Browser / GUI agents Efficient action spaces、long-horizon computer interaction、安全 ACI
Skills & Composition Skills、workflows、multi-agent Reusable capability、skill lifecycle、agent composition
Protocol & Interoperability MCP、A2A large ecosystem、discovery、identity、security、task protocols
Runtime & Safety Sandbox、guardrails、tracing Durable runtime、containment、permission、agent security、cost/reliability

3.3 Benchmark

3.3.1.1 Generalist / Long-Horizon Agent Benchmark

The agent workflow nowadays basically is:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
Goal
 ↓
Planning
 ↓
Tool calls
 ↓
Environment interaction
 ↓
State changes
 ↓
Observation
 ↓
Recovery
 ↓
More tool calls
 ↓
Final artifact

The tradition benchmark only judge if the final answer is correct.

This research direction is about:

1
Can the Agent still complete the task after taking dozens, hundreds, or even thousands of actions?

  • TheAgentCompany, NeurIPS 2025

    This Benchmark builds a company system:

    1
    2
    3
    4
    5
    6
    7
    8
    Company Environment
    │
    ├── Browser
    ├── Code repository
    ├── Terminal
    ├── Internal websites
    ├── Communication
    └── Coworkers

  • AgencyBench, ACL 2026

  • GAIA — NeurIPS 2023

3.3.1.2 Computer-Use / GUI Agent Benchmark

1
Can Agent really operate computer instead of discribing operations?

  • OSWorld, NeurIPS 2024.

    The most important Computer-Use Benchmark.

    1
    2
    3
    4
    5
    6
    7
    OSWorld
    ├── Desktop OS Environment
    ├── Browser
    ├── Office Applications
    ├── File System
    ├── Terminal
    └── Other Desktop Applications

  • OSWorld-Human, ICML 2025

    Calculate efficiency of Agent.

    Even when they successfully complete the task, top agents still require 1.4–2.7× as many steps as humans.

  • OSWorld 2.0

  • CRAB, ACL 2025

    Cross-Environment Agent Benchmark for Multimodal Language Model

    Thought: Benchmark should not be bound by certain environment.

3.3.1.3 Tool Use / Function Calling / MCP Benchmark

Tools Evaluation Process:

1
2
3
4
5
6
7
8
9
Function Calling
      ↓
Tool Use
      ↓
Stateful Tool Use
      ↓
Agentic Tool Use
      ↓
MCP / Interoperability

  • BFCL, ICML 2025

    BFCL v4

    It judges:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    Single-turn
       ↓
    Multiple functions
       ↓
    Parallel calls
       ↓
    Multi-turn
       ↓
    Memory
       ↓
    Dynamic decision making
       ↓
    Long-horizon agentic behavior

  • ToolSandbox, NAACL 2025 Findings

  • τ-bench, ICLR 2025

  • MCP-Bench, ICLR 2026

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    Tool Discovery
          ↓
    Tool Selection
          ↓
    Parameter Control
          ↓
    Multi-hop Planning
          ↓
    Cross-tool Coordination
          ↓
    Task Completion

3.3.1.4 Software Engineering Agent Benchmark

  • SWE-bench, ICLR 2024 Oral

  • SWE-bench Multimodal, ICLR 2025

  • SWE-Bench Pro, ICLR 2026 submission.

    For Long-Horizon software engineering.

There’s a problem: ==Contamination==.

The paper finds that some models’ performance on SWE-bench Verified may be affected by training data contamination:

Does SWE-Bench-Verified Test Agent Ability or Model Memory?

3.3.1.5 Web / Browser Agents Benchmark

The core of Web Agent is not ‘Search’ but: The Agent observes webpages, plans, clicks, types, navigates, calls APIs, and ultimately completes tasks in real-world web environments.

Classic workflow:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
User Goal
   ↓
Browser
   ↓
Observe webpage
   ↓
Plan
   ↓
Click / Type / Search
   ↓
New webpage
   ↓
Reason
   ↓
...
   ↓
Task Success
  • WebArena, ACL 2024
  • TurkingBench, NAACL 2025
  • X-WebAgentBench, ACL 2025 Findings

3.3.1.6 Deep Research / Research Agent Benchmark

Normal Web Research:

1
2
3
4
5
6
7
8
9
Web Agent

Goal
 ↓
Browse
 ↓
Action
 ↓
Done

Deep Research:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
Research Question
 ↓
Decompose
 ↓
Search
 ↓
Read
 ↓
Cross-check
 ↓
Identify information gaps
 ↓
Search again
 ↓
Synthesize
 ↓
Cite sources
 ↓
Report

  • BrowseComp

  • BrowseComp-Plus, ACL 2026

  • MLR-Bench, NeurIPS 2025:

    1
    2
    3
    4
    5
    Web Research
          ↓
    Deep Research
          ↓
    Scientific Research Agent

    Can AI do research itself?

3.3.1.7 Benchmark Methodology

  • BrowseComp-Plus, ACL 2026

3.3.2.1 SWE-bench / SWE-bench Verified

3.3.2.2 GAIA

3.3.2.3 WebArena

3.3.2.4 OSWorld Verified/OSWorld 2.0

3.3.2.5 AgentBench

It’s a benchmark suite including:

1
2
3
4
5
6
7
8
9
10
AgentBench
│
├── OS
├── Database
├── Knowledge Graph
├── Digital Card Game
├── Lateral Thinking Puzzle
├── ALFWorld
├── WebShop
└── Mind2Web