ALFWorld

Posted on Edited on In Views:
About ALFWorld Benchmark

1. Paper Reading

2. Engineering

2.1 Introduction

2.1.1 Brief

Repo URL: ALFWorld

ALFWorld is a Text-based interaction household task benchmark.

Agent/LLM may receive a text-descripted task:

1
2
Your task is to:
put a clean apple in the fridge.

The Environment will feedback continually:

1
2
3
4
You are in the kitchen.
You see a countertop 1.
You see a fridge 1.
...

e.g.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
Agent Step 1:
	go to countertop 1
Envrionment Feedback:
	You arrive at countertop 1.
	You see an apple 1.

Agent Step 2:
	take apple 1
Envrionment Feedback:
	You pick up the apple 1.

Agent Step 3:
	go to fridge 1
Envrionment Feedback:
	Your arrive at fridge 1

Agent Step 4:
	put apple 1 in fridge 1
Envrionment Feedback:
	You Won!

2.1.2 Virtual Envs

ALFWorld supports 2 kinds of test: TextWorld and THOR.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
                   ALFWorld
                      │
       ┌──────────────┴──────────────┐
       │                             │
       ▼                             ▼
  TextWorld                       THOR
AlfredTWEnv                  AlfredThorEnv
       │                             │
       ▼                             ▼
Text Observation                  Images
       │                             │
       ▼                             ▼
 Text LLM Agent               Vision Agent
       │                             │
       ▼                             ▼
 Planning / RL /              Embodied /
 Memory / ReAct               Visual Control

2.2 Task Structure

Download task dataset (Windows):

1
2
3
4
5
6
7
8
9
10
conda create -n alfworld python=3.10
conda activate alfworld
python -m pip install requests tqdm
export PYTHONPATH="$PWD"
echo $PYTHONPATH
python -c "import alfworld; print(alfworld.__file__)"
python -c "from alfworld.info import ALFWORLD_DATA; print(ALFWORLD_DATA)"
mkdir -p /e/Agent/Datasets/ALFWorld

PYTHONPATH="$PWD" python scripts/alfworld-download     --data-dir /e/Agent/Datasets/ALFWorld
1
2
3
4
5
6
7
8
9
10
11
12
13
14
test@▒ѩ▒ MINGW64 /e/Agent/Datasets
$ tree -A -a -L 3
.
└── ALFWorld
    ├── detectors
    │   └── mrcnn_alfred_objects_sep13_004.pth
    ├── json_2.1.1
    │   ├── train
    │   ├── valid_seen
    │   ├── valid_train
    │   └── valid_unseen
    └── logic
        ├── alfred.pddl
        └── alfred.twl2

The main tasks are 2 kinds: valid_seen and valid_unseen

A concrete instance:

1
2
3
4
5
6
7
test@ѩ▒ MINGW64 /e/Agent/Datasets/ALFWorld/json_2.1.1/valid_seen/look_at_obj_in_light-AlarmClock-None-DeskLamp-323
$ tree -A
.
└── trial_T20190909_044715_250790
    ├── game.tw-pddl
    ├── initial_state.pddl
    └── traj_data.json

2.2.1 ALFRED & AI2-THOR

ALFRED(Action Learning From Realistic Environments and Directives), is a benchmark for ==Embodied Agent==, based on AI2-THOR.

The benchmark is to solve the kind of problem:

Give the Agent a natural-language instruction and have it complete the task through a sequence of actions in a virtual 3D household environment.

e.g. 

1
2
3
Instruction:

"Put the clean mug in the cabinet."

Agent:

1
2
3
4
5
6
7
Kitchen
  ├── Find the mug
  ├── Find the cabinet
  ├── Pick up the mug
  ├── Walk to the cabinet
  ├── Open the cabinet
  └── Put the mug inside

The ==core feature== of ALFRED is:

Demonstrations of humans actually completing tasks in the AI2-THOR environment.

AI2-THOR demonstration:

1
2
3
4
5
6
7
8
Human Expert
      │
      │ Complete the task
      ▼
AI2-THOR 3D Environment
      │
      ▼
Expert Trajectory

e.g.

  • Task

    1
    Put the alarm clock in the drawer.

  • Expert Trajectory:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    Step 0
    Observation:
    Bedroom

    Action:
    go to desk


    Step 1
    Observation:
    Alarm clock is on desk

    Action:
    pickup alarm clock


    Step 2
    Observation:
    Holding alarm clock

    Action:
    go to drawer


    Step 3
    Observation:
    Drawer is closed

    Action:
    open drawer


    Step 4
    Observation:
    Drawer is open

    Action:
    put alarm clock in drawer


    Step 5
    Goal achieved

The task data of ALFWorld comes from ALFRED:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
                 ALFRED
                    │
           Human Expert Demo
                    │
                    ▼
           Expert Trajectory
                    │
   ┌────────────────┼────────────────┐
   │                │                │
   ▼                ▼                ▼
Images            Actions          States
   │                │                │
   └────────────────┼────────────────┘
                    │
                    ▼
             ALFRED Task Data
                    │
                    ▼
             ALFWorld Processing
                    │
      ┌─────────────┴─────────────┐
      ▼                           ▼
 TextWorld                    AI2-THOR
      │                           │
      ▼                           ▼
Text Observation            Visual Observation
      │                           │
      ▼                           ▼
  Your Agent                  Your Agent
      │                           │
      └─────────────┬─────────────┘
                    ▼
             Agent Trajectory
                    │
                    ▼
             Goal Satisfied?
                    │
              ┌─────┴─────┐
              ▼           ▼
           Success      Failure

2.2.2 traj_data.json

To describe what the task is, including its task type, goal parameters, scene, expert trajectory, etc.

The traj_data.json can be divided into following parts:

1
2
3
4
5
6
7
8
traj_data.json
└───────
	   ├── images
	   ├── pddl_params
	   ├── plan
	   ├── scene
	   ├── turk_annotations
	   └── task_id & task

2.2.2.1 images

  • images structure:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    "images": [
        	...
    		{
    			"high_idx": 0,
    			"image_name": "000000000.png",
    			"low_idx": 0
    		}
    		...
    ]

2.2.2.2 pddl_params

  • pddl_params structure:

    PDDL(Planning Domain Definition Language) is used to generate a text world.

    1
    2
    3
    4
    5
    6
    7
    "pddl_params": {
            "mrecep_target": "",
            "object_sliced": false,
            "object_target": "AlarmClock",
            "parent_target": "",
            "toggle_target": "DeskLamp"
    }
    • mrecep_target: movable receptacle
    • object_sliced: If object should be sliced
    • object_target: target object
    • parent_target: parent target
    • toggle_target: The switch should be toggled

2.2.2.3 plan

  • plan main structure:

    1
    2
    3
    4
    "plan": {
        "high_pddl": [...],
        "low_actions": [...]
    }
    • high_pddl is to describe high-level of expert trajectory. e.g .
      • GotoLocation / PickupObject / PutObject /…
    • low_actions is to describe concrete actions e.g.
      • LookDown / Forward / RotateLeft /

  • high_pddl and low_actions are corresonding

    e.g.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    high_pddl[0]
        │
        ├── low_actions[0]
        ├── low_actions[1]
        ├── low_actions[2]
        ├── ...
        └── low_actions[n]

    high_pddl[1]
        │
        ├── low_actions[n+1]
        ├── low_actions[n+2]
        └── ...
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    high_pddl
    │
    ├── [0] GotoLocation(dresser)
    │       │
    │       │ high_idx = 0
    │       ▼
    │   low_actions
    │       ├── LookDown_15
    │       ├── RotateLeft_90
    │       ├── MoveAhead
    │       ├── MoveAhead
    │       └── ...
    │
    ├── [1] PickupObject(mug)
    │       │
    │       │ high_idx = 1
    │       ▼
    │   low_actions
    │       ├── ...
    │       └── PickupObject
    │
    └── [2] ...

  • A action description may includes many visual information but TextWorld do not need:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    {
        "api_action": {
            "action": "ToggleObjectOn",
            "objectId": "DeskLamp|..."
        },
        "discrete_action": {
            "action": "ToggleObjectOn",
            "args": {
                "bbox": [...],
                "mask": [...],
                "point": [88, 111]
            }
        },
        "high_idx": 3
    }

2.2.2.4 scene

“==scene==” is to what is the state of the world when the task begins.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
"scene": {
    "dirty_and_empty": false,
    "floor_plan": "FloorPlan323",

    "init_action": {
        "action": "TeleportFull",
        "horizon": 30,
        "rotateOnTeleport": true,
        "rotation": 90,
        "x": 0.25,
        "y": 0.9009992,
        "z": 1.0
    },

    "object_poses": [...],
    "object_toggles": [...],

    "random_seed": 3619629972,
    "scene_num": 323
}

2.2.2.5 turk_annotations

1
2
3
4
5
turk_annotations
└── anns
    ├── annotation 1
    ├── annotation 2
    └── annotation 3

For one same task, there may exists many different annotations:

e.g. 

1
Look at an alarm clock by the light of a lamp.

1
Look at the clock under the lamp.

For a concrete annotations:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
{
                "assignment_id": "A3UF6XXFFRR237_3I0BTBYZA0CU1NAVSL6DC5BBKB0Y0X",
                "high_descs": [
                    "go to the table with the safe ",
                    "pick up the clock",
                    "take the clock to the desk",
                    "turn on the lamp"
                ],
                "task_desc": "look at the clock under the lamp",
                "votes": [
                    1,
                    1,
                    1,
                    1,
                    1
                ]
            }

2.2.3 initial_state.pddl

Describe the initial world state of the task.

The initial_state.pddl can be divided into 3 main parts:

1
2
3
4
5
6
7
8
9
10
initial_state.pddl
│
├── 1. :objects
│      └── Define: "What do the Env have"
│
├── 2. :init
│      └── Define: "What is the state of Env now"
│
└── 3. :goal
       └── Define: "Final State of the Env"

e.g.

  • object:

    1
    2
    3
    4
    5
    6
    agent1 - agent
    Mug_xxx - object
    MugType - otype
    Desk_xxx - receptacle
    DeskType - rtype
    loc_xxx - location

  • init:

    1
    2
    3
    4
    (atLocation agent1 loc_xxx)
    (inReceptacle Mug_xxx Desk_xxx)
    (pickupable Mug_xxx)
    (openable Drawer_xxx)

  • goal:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    (:goal
                 (and
                     (exists (?ot - object
                              ?r - receptacle
                              ?a - agent
                              ?l - location)
                         (and
                             (objectType ?ot DeskLampType)
                             (toggleable ?ot)
                             (isToggled ?ot)
                             (receptacleAtLocation ?r ?l)
                             (atLocation ?a ?l)
                             (inReceptacle ?ot ?r)
                         )
                     )
                     (exists (?o - object
                              ?a - agent)
                         (and
                             (objectType ?o AlarmClockType)
                             (holds ?a ?o)
                         )
                     )
                 )
            )

2.2.4 game.tw-pddl

Convert the task into game rules that can be executed by TextWorld.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
game.tw-pddl
│
├── pddl_domain
│      └── Defines the "game rules"
│
├── grammar
│      └── Defines the interface between "PDDL ↔ natural language"
│
├── pddl_problem
│      └── Defines the "world, initial state, and goal" of this specific task
│
├── solvable
│      └── Indicates whether this task is solvable
│
└── walkthrough
       └── A known feasible expert solution path

The game.tw-ddl is composed of different parts:

1
python -c "import json; d=json.load(open('game.tw-pddl', encoding='utf-8'), strict=False); open('domain.pddl','w',encoding='utf-8').write(d['pddl_domain']); open('problem.pddl','w',encoding='utf-8').write(d['pddl_problem']); open('grammar.txt','w',encoding='utf-8').write(str(d['grammar'])); open('walkthrough.txt','w',encoding='utf-8').write(str(d['walkthrough'])); print('solvable =', d['solvable'])"

2.2.5 Summary

File Function Child Item
traj_data.json Task / Expert Trajectory task、pddl_params、scene、plan、turk_annotations、images …
initial_state.pddl ****The symbolic state and goal of the task**** :objects、:init、:goal
game.tw-pddl Change Task into TextWorld Game pddl_domain、pddl_problem、grammar、solvable、walkthrough

2.3 Task Workflow

2.3.1 Entrance

alfworld/scripts/run_eval.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
def run_eval():
    # Load Config
    config = generic.load_config()
    # Create Agent
    agent = TextDAggerAgent(config)
	# Create Output Direction
    output_dir = os.path.expandvars(config["general"]["save_path"])
    if not os.path.exists(output_dir):
        os.makedirs(output_dir)

    # load trained model from checkpoint
    data_dir = os.path.expandvars(config["general"]["save_path"])
    if agent.load_pretrained:
        print("Checking {} for {}".format(data_dir, agent.load_from_tag))
        if os.path.exists(data_dir + "/" + agent.load_from_tag + ".pt"):
            agent.load_pretrained_model(data_dir + "/" + agent.load_from_tag + ".pt")
            agent.update_target_net()

	# Dagger & DQN
    training_method = config["general"]["training_method"]
    # Testdata Paths
    eval_paths = config["general"]["evaluate"]["eval_paths"]
    #TextWorld or Thor
    eval_envs = config["general"]["evaluate"]["envs"]
    # for Thor
    controllers = config["general"]["evaluate"]["controllers"]
    # repeat times
    repeats = config["general"]["evaluate"]["repeats"]

    # iterate through all environments
    for eval_env_type in eval_envs:
        # iterate through all controllers
        for controller_type in (controllers if eval_env_type == "AlfredThorEnv" else ["tw"]):
            print("Setting controller: %s" % controller_type)
            # iterate through all splits
            for eval_path in eval_paths:
                print("Evaluating: %s" % eval_path)
                config["general"]["evaluate"]["env"]["type"] = eval_env_type
                config["dataset"]["eval_ood_data_path"] = eval_path
                config["controller"]["type"] = controller_type

                alfred_env = get_environment(config["general"]["evaluate"]["env"]["type"])(config, train_eval="eval_out_of_distribution")
                eval_env = alfred_env.init_env(batch_size=agent.eval_batch_size)

                # evaluate method
                if training_method == "dagger":
                    results = evaluate_dagger(eval_env, agent, alfred_env.num_games*repeats)
                elif training_method == "dqn":
                    results = evaluate_dqn(eval_env, agent, alfred_env.num_games*repeats)
                else:
                    raise NotImplementedError()

                # save results to json
                split_name = eval_path.split("/")[-1]
                experiment_name = config["general"]["evaluate"]["eval_experiment_tag"]
                results_json = os.path.join(output_dir, "{}_{}_{}_{}.json".format(experiment_name, eval_env_type.lower(), controller_type, split_name))

                with open(results_json, 'w') as f:
                    json.dump(results, f, indent=4, sort_keys=True)
                print("Saved %s" % results_json)

                eval_env.close()

2.3.2 Collect Game List

alfworld/alfworld/agents/environment/alfred_tw_env.py:118

1
2
3
4
5
6
7
8
9
10
11
def __init__(self, config, train_eval="train"):
    print("Initializing AlfredTWEnv...")
    self.config = config
    self.train_eval = train_eval

    if config["env"]["goal_desc_human_anns_prob"] > 0:
        msg = ("Warning! Changing `goal_desc_human_anns_prob` should be done with"
               " the script `alfworld-generate`. Ignoring it and loading games as they are.")
        print(colored(msg, "yellow"))

    self.collect_game_files()

2.3.3 Start TextWorld

alfworld/alfworld/agents/environment/alfred_tw_env.py:245

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
  def init_env(self, batch_size):
      # Randomization is available when training
      # not available when evaluating
      domain_randomization = self.config["env"]["domain_randomization"]
      if self.train_eval != "train":
          domain_randomization = False
# ...
      
      # Register a new Gym environment.
      # Hope environment return info including 'won'
      # admissible_commands
      request_infos = textworld.EnvInfos(won=True, admissible_commands=True, extras=["gamefile"])

# ...
      
      env_id = textworld.gym.register_games(self.game_files, request_infos,
                                            batch_size=batch_size,
                                            asynchronous=True,
                                            max_episode_steps=max_nb_steps_per_episode,
                                            wrappers=wrappers)
      # Launch Gym environment.
      env = textworld.gym.make(env_id)
      return env

Core code:

1
2
3
4
5
6
7
8
env_id = textworld.gym.register_games(
    self.game_files,
    request_infos,
    batch_size=batch_size,
    asynchronous=True,
    max_episode_steps=max_nb_steps_per_episode,
    wrappers=wrappers
)

game_files is the set of game.tw-pddl

Gym is a standardized environment for agents to interact with.

2.3.4 Evaluation

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
def evaluate_dagger(env, agent, num_games, debug=False):
    # fixed seed, 42
    env.seed(42)
    
    # obs = observation, infos = extra info
    obs, infos = env.reset()
    
    # current test item
    game_names = infos["extra.gamefile"]
    
    # reset agent's information
    agent.init(batch_size)
    
    # previous action + current observation
    execute_actions = []
	...
	execute_actions.append("restart")
    
    # task description
    observation_strings = list(obs)

	task_desc_strings, observation_strings = \
    agent.get_task_and_obs(observation_strings)

	task_desc_strings = agent.preprocess_task(task_desc_strings)
	observation_strings = agent.preprocess_observation(observation_strings)

Input for Agent like:

1
2
3
4
5
6
Task:
look at the alarm clock under the lamp

Observation:
You are in the bedroom.
You see a dresser...
1
action_candidate_list = list(infos["admissible_commands"])

Start Loop:

1
2
3
4
for step_no in range(agent.max_nb_steps_per_episode):
    # ...
    obs, _, dones, infos = env.step(execute_actions)
    # ...