Wall-OSS-0.5

Wall-OSS-0.5 is a 4B vision-language-action model from XSquare Robot. It combines a Qwen2.5-VL vision-language backbone with a continuous flow-matching action expert.

LeRobot provides Wall-OSS-0.5 through the wall_oss_05 policy type and the lerobot/wall-oss-0.5 checkpoint.

wall_oss_05 and the existing wall_x policy target different WALL releases. Use wall_oss_05 with the Wall-OSS-0.5 checkpoint.

Model Overview

Wall-OSS-0.5 conditions action generation on camera observations, robot state, and a natural-language task. The policy predicts a chunk of continuous actions with flow matching. Its vision-language expert can also generate text for captioning, visual question answering, grounding, and language subtasks.

The released model uses:

• a 26-dimensional canonical state and action representation;
• 32-step action chunks;
• q01/q99 range normalization;
• a 10-step flow sampler;
• Qwen2.5-VL for visual and language inputs.

The base checkpoint is intended for fine-tuning on a target robot and task dataset before deployment.

What the LeRobot Integration Covers

• Standard policy.type=wall_oss_05 configuration and Hub checkpoint loading.
• Native Qwen2.5-VL vision-language processing and Wall action experts.
• Continuous x-prediction flow training and 10-step Euler sampling.
• Recipe-driven text cross-entropy and joint text/action fine-tuning.
• Native text generation for VQA, grounding, captions, and language subtasks.
• Direct text generation through LeRobot’s interactive rollout.
• Canonical 26D state and action inputs using LeRobot q01/q99 processors.
• Fine-tuning with lerobot-train and inference through standard policy APIs.

Installation Requirements

Wall-OSS-0.5 requires an NVIDIA GPU with CUDA. Install LeRobot with the policy dependencies:

pip install "lerobot[wall_oss_05]"

For a LeRobot source checkout:

uv sync --extra wall_oss_05

The policy is implemented natively in LeRobot. Its only policy-specific dependency is Transformers; no separate Wall-X checkout is required. Recipe-backed processing also requires the dataset extra. For language queries or training, install uv sync --extra wall_oss_05 --extra training (training includes dataset support). Set --policy.recipe=null for action-only processing without the dataset extra.

Checkpoints

The checkpoint includes the policy configuration, Qwen processor and tokenizer, model weights, saved LeRobot pre/postprocessors, normalization statistics, and conversion metadata. It is a base checkpoint: fine-tune it on the target robot and task before reporting task-level results.

Usage

Select the policy with:

--policy.type=wall_oss_05

Initialize from the LeRobot checkpoint with:

--policy.path=lerobot/wall-oss-0.5

Use --policy.path=your-org/your-finetuned-checkpoint when evaluating or deploying a saved LeRobot checkpoint.

Text and VQA

Wall-OSS-0.5 uses the shared policy text contract. Runtime supplies query_kind and query_text, then runs the normal policy input processor:

request = {
    **observation,
    "query_kind": "vqa",
    "query_text": "Which object is closest to the right gripper?",
}
processed_batch = preprocessor(request)
answer = policy.generate_text(processed_batch)

vqa preserves the caller question. next_subtask applies the recipe owned by WallOSS05Config. The policy inherits generate_text(processed_batch) and overrides supports_text_generation and generate_text. Generation uses the checkpoint’s tied Qwen vocabulary head and vision-language expert; it does not call a separate VLM.

The language runtime can generate a subtask and then condition Wall’s action prediction on it:

lerobot-rollout \
  --policy.path=your-org/your-finetuned-wall-oss-0.5 \
  --policy.device=cuda \
  --task="clear the table" \
  --interactive=true

After /start, use /vqa <question>, /autosteer <goal>, or /subtask <text>.

Wall retains useful embodied VQA and grounding capabilities, but it is specialized for robot observations and is not intended to replace a general-purpose VLM.

Data Requirements

Wall-OSS-0.5 operates internally on a canonical 26D state and action layout. Narrower state and action features are normalized with their native dataset statistics, then zero-padded to 26D for the model. Padded action dimensions are excluded from the training loss. Outputs are cropped to the dataset’s action width before unnormalization: an 8D state with 7D actions returns 7D actions. postprocess_action_dim can apply an additional output crop. Padding does not remap joint coordinates or controller conventions; supply compatible values.

Configure image inputs with camera_key_mapping. The mapping associates each LeRobot image feature with a Wall camera slot. The default checkpoint uses face_view and right_wrist_view.

The dataset task string is passed to the policy as its language instruction. Use consistent task descriptions when recording, training, and evaluating.

State and action q01/q99 normalization is handled by the saved LeRobot preprocessor and postprocessor. Inference restores the statistics saved with the checkpoint. When fine-tuning with lerobot-train, the training dataset statistics replace the pretrained processor statistics and are saved with the new checkpoint.

The published checkpoint uses the native LeRobot policy state dictionary and current pre/postprocessor format. Weights load and save through the shared policy persistence code, including distributed gathers and the main-process write gate. The config retains a stable tokenizer asset reference and can be loaded from any working directory.

Training

Fine-tune the base checkpoint with the standard LeRobot training command:

Training Command Example

lerobot-train \
  --dataset.repo_id=your-org/your-dataset \
  --policy.path=lerobot/wall-oss-0.5 \
  --policy.repo_id=your-org/wall-oss-0.5-your-task \
  --output_dir=outputs/train/wall_oss_05 \
  --job_name=wall_oss_05_finetune \
  --policy.device=cuda \
  --batch_size=1 \
  --steps=20000

Start with a small batch size because Wall-OSS-0.5 is a 4B-parameter model. Increase it according to available GPU memory, or use gradient accumulation through your distributed training configuration.

For datasets with different camera names, map their feature keys to the Wall camera slots. For example, add the following option for top and wrist:

--policy.camera_key_mapping='{"observation.images.top":"face_view","observation.images.wrist":"right_wrist_view"}'

Fine-tuning resolves camera shapes and state/action widths from the dataset, including when loading the base checkpoint with --policy.path.

Train with language annotations

The recipe owned by WallOSS05Config uses LeRobot’s optional language columns by default. Its joint subtask recipe supervises the assistant response and continuous action chunk in one causal sequence:

lerobot-train \
  --dataset.repo_id=your-org/your-language-annotated-dataset \
  --policy.path=lerobot/wall-oss-0.5 \
  --policy.use_language_recipe=true \
  --policy.text_loss_weight=0.1 \
  --policy.flow_loss_weight=1.0 \
  --policy.device=cuda \
  --batch_size=1 \
  --steps=20000

Recipe streams determine which objectives run for each sample:

• a targeted assistant turn receives next-token text cross-entropy;
• a low_level turn receives continuous flow-action supervision; and
• a low_level recipe with a targeted assistant turn receives both losses, with the action tokens attending to the supervised text.

Only the targeted assistant payload and its <|im_end|> token are used as language labels. User prompts, padding, images, proprioception, and continuous action-query tokens are masked from text cross-entropy. Rows without annotations retain the original action-only prompt and loss. Set use_language_recipe=false to disable annotation rendering, or recipe=null to disable both recipe renderers. An explicit recipe_path overrides the built-in recipe. Fresh fine-tuning rebuilds processors from that recipe and the training statistics; resumed training and rollout restore the checkpoint’s saved processors and statistics.

See Language columns and recipes for the annotation schema and recipe structure. recipe_path may point to an explicit external YAML override; LeRobot does not provide shared default recipe files.

Key Configuration Parameters

Evaluation and Inference

Use a fine-tuned checkpoint with the standard LeRobot evaluation or robot rollout workflows by passing its Hub repository or local directory as the policy path.

Wall-OSS-0.5 generates continuous action chunks with its flow sampler. The current policy supports the continuous flow branch; the autoregressive action branch is not exposed.

Each chunk encodes its camera images and prefills the observation/text prefix once. The ten Euler steps reuse the prefix keys and values and decode only the action tokens. The cache is rebuilt for each new observation.

For a supported simulator, provide the fine-tuned checkpoint and the environment-specific options through lerobot-eval:

lerobot-eval \
  --policy.path=your-org/your-finetuned-wall-oss-0.5 \
  --policy.device=cuda \
  --env.type=<environment> \
  --env.task=<task> \
  --eval.batch_size=1 \
  --eval.n_episodes=10

Do not evaluate the base checkpoint as though it were a task-specific policy. The canonical representation, normalization statistics, and task distribution must match the fine-tuned checkpoint.

References

• Wall-OSS-0.5 project page
• Wall-OSS-0.5 LeRobot checkpoint

License

Wall-OSS-0.5 and Wall-X are released under the Apache 2.0 License.