Train robotics AI models

The phospho dev kit makes it easy to train robotics AI models by integrating with LeRobot from Hugging Face.

​

What is LeRobot?

LeRobot is a platform designed to make real-world robotics more accessible for everyone. It provides pre-trained models, datasets, and tools in PyTorch.

It focuses on state-of-the-art approaches in imitation learning and reinforcement learning.

With LeRobot, you get access to:

• Pretrained models for robotics applications
• Human-collected demonstration datasets
• Simulated environments to test and refine AI models

Useful links:

• LeRobot on GitHub
• LeRobot on Hugging Face
• AI models for robotics

​

Step by step guide

In this guide, we will use the phospho Junior dev kit to record a dataset and upload it to Hugging Face.

​

Prerequisites

• A Hugging Face account to automatically upload your dataset (Sign up here if you don’t have one)
• A device for training your model. We recommend using a GPU for faster training.
• The control module powered on and connected to the robot arm and camera.
• The phospho teleo app installed on your Meta Quest 2, Pro, 3 or 3s

​

1. Set up your Hugging Face token

To upload datasets, you need a Hugging Face token with write access. Follow these steps to generate one:

1. Log in to your Hugging Face account

2. Go to Profile and click Access Tokens in the sidebar.

3. Select the Write option to grant write access to your account. This is necessary for creating new datasets and uploading files. Name your token and click Create token.

4. Copy the token and save it in a secure place. You will need it later.

5. Open a browser and access phosphobot.local:80/admin to access the control module server.

6. Paste the Hugging Face token, and save it.

​

2. Record a dataset by teleoperating the robot

1. Go to the Meta Quest Store, search for the Phospho Teleop application.

2. Open the app. You should see a screen displaying phoshobot along with the server ping. Click Connect.
   

   Ensure that you are connected to the same network as the control module.

3. After connecting, you should see the main camera feed if a camera is connected. You can move various assets in the metaverse as needed.

   • Press A once to start teleoperation. Move your controller in the space to move the robot.
   • Press B to start recording.
   • Press B again to stop the recording.
   • Press A again to stop teleoperation. The robot will stop moving.

4. Control the robot arm. Start the recording. Perform the manipulation. Stop the recording when you’re done.
   The recording is automatically saved in LeRobot v2 format and uploaded to your Hugging Face account. You can check it in your profile.

To change the dataset name and recording settings, go to your admin panel and go to the Admin settings page.

​

3. Train your first model

1. On the device where you want to run the training, install the Phosphobot package:

   pip install --upgrade phosphobot
   

2. Clone the LeRobot repository and install it:

   git clone https://github.com/huggingface/lerobot.git
   cd lerobot
   pip install -e .  # Requires Python 3.10+
   

3. (Optional) If you want to use Weights & Biases for tracking training metrics, log in with:

   wandb login
   

4. Run the training script with the following command in the lerobot repository (Set —device=mps for Apple Silicon (Mac M1/M2), cuda if you have an NVIDIA GPU or cpu if you don’t have a GPU). Ensure that your lerobot virtual environment is activated.

   sudo python lerobot/scripts/train.py \
     --dataset.repo_id=<HF_USERNAME>/<DATASET_NAME> \
     --policy.type=<act or diffusion or tdmpc or vqbet> \
     --output_dir=outputs/train/phoshobot_test \
     --job_name=phosphobot_test \
     --device=mps \
     --wandb.enable=true
   

5. Your trained model will be saved in lerobot/outputs/train/.

6. For inference scripts, you can head to our opensource GitHub repo, under the inference folder: here

​

What’s next?

• Join our Discord and send us a video of your robot! 💚
• Share your trained models on Hugging Face and help others get started with robotics AI.
• Explore the rest of the documentation and try out the example scripts.