The phospho starter pack makes it easy to train robotics AI models by integrating with LeRobot from Hugging Face.

​
What is LeRobot?

LeRobot is a platform designed to make real-world robotics more accessible for everyone. It provides pre-trained models, datasets, and tools in PyTorch.

It focuses on state-of-the-art approaches in imitation learning and reinforcement learning.

With LeRobot, you get access to:

  • Pretrained models for robotics applications
  • Human-collected demonstration datasets
  • Simulated environments to test and refine AI models

Useful links:

​
Step by step guide

In this guide, we will use the phospho starter pack to record a dataset and upload it to Hugging Face.

Before starting, make sure you have already completed the Installation Guide.

​
Prerequisites

  • A Hugging Face account to automatically upload your dataset (Sign up here if you don’t have one)
  • A device for training your model. We recommend using a GPU for faster training.
  • The phosphobot server is running and connected to the robot arm and camera.
  • The phosphobot teleoperation app is installed on your Meta Quest 2, Pro, 3 or 3s

​
1. Set up your Hugging Face token

To upload datasets, you need a Hugging Face token with write access. Follow these steps to generate one:

  1. Log in to your Hugging Face account

  2. Go to Profile and click Access Tokens in the sidebar.

  3. Select the Write option to grant write access to your account. This is necessary for creating new datasets and uploading files. Name your token and click Create token.

  1. Copy the token and save it in a secure place. You will need it later.

  2. Make sure the phosphobot server is running. Open a browser and access localhost or phosphobot.local if you’re using the control module. Then go to the Admin settings.

  3. Paste the Hugging Face token, and save it.

​
2. Record a dataset by teleoperating the robot

Make sure you have already completed the Installation Guide and that you have a Meta Quest

  1. Go to the Meta Quest Store, search for the Phospho Teleop application.

  2. Open the app. You should see a screen displaying phoshobot along with the server ping. Click Connect.

    Ensure that you are connected to the same network as the phosphobot server.

  1. After connecting, you should see the main camera feed if a camera is connected. You can move various assets in the metaverse as needed.

    • Press A once to start teleoperation. Move your controller in the space to move the robot.
    • Press B to start recording.
    • Press B again to stop the recording.
    • Press A again to stop teleoperation. The robot will stop moving.

  2. Control the robot arm. Start the recording. Perform the manipulation. Stop the recording when you’re done.
    The recording is automatically saved in LeRobot v2 format and uploaded to your Hugging Face account. You can check it in your profile.

To change the dataset name and recording settings, go to your admin panel and go to the Admin settings page.

​
3. Train your first model

​
Cloud training

To train your model, you can use Replicate. Replicate provides GPU instances to train AI models in one click.

Train ACT on Replicate

The ACT model mimicks one behavior at a time

Train gr00t-n1 on Replicate

The gr00t-N1 model is a promptable model by NVIDIA

​
Running on your local machine

You need a GPU with at least 16GB of memory to train the model.

  1. On the device where you want to run the training, install the Phosphobot package:

    pip install --upgrade phosphobot

  2. Clone the LeRobot repository and install it:

    git clone https://github.com/huggingface/lerobot.git
cd lerobot
pip install -e .  # Requires Python 3.10+

  3. (Optional) If you want to use Weights & Biases for tracking training metrics, log in with:

    wandb login

  4. Run the training script with the following command in the lerobot repository (Set —device=mps for Apple Silicon (Mac M1/M2), cuda if you have an NVIDIA GPU or cpu if you don’t have a GPU). Ensure that your lerobot virtual environment is activated.

    sudo python lerobot/scripts/train.py \
  --dataset.repo_id=<HF_USERNAME>/<DATASET_NAME> \
  --policy.type=<act or diffusion or tdmpc or vqbet> \
  --output_dir=outputs/train/phoshobot_test \
  --job_name=phosphobot_test \
  --device=mps \
  --wandb.enable=true

  5. Your trained model will be saved in lerobot/outputs/train/.

  6. For inference scripts, you can head to our opensource GitHub repo, under the inference folder: here

​
What’s next?

  • Join our Discord and send us a video of your robot! 💚
  • Share your trained models on Hugging Face and help others get started with robotics AI.
  • Explore the rest of the documentation and try out the example scripts.

Was this page helpful?