Seeed Studio reBot Arm B601-RS and NVIDIA Jetson AGX Thor: Building a Physical AI Pipeline

Overview​

I want to learn how to train my Seeed Studio reBot Arm B601-RS, and learn robotics properly along the way, using my NVIDIA Jetson AGX Thor to do it.

The model is NVIDIA Isaac GR00T N1.7. The Thor is where I want all of it to run — training and running the model both.

This is physical AI: AI that controls a real machine — here, a robot arm learning a task from examples. I train it in a loop, the flywheel: record the arm doing a task by moving it with a second "leader" arm (teleoperation), train the model on those recordings, test it on the real arm, make more varied recordings from them, and train again. Each lap should make the model better.

Train stage: I tested that first, and it works. 10,000 steps in 6 h 34 m, about NZ$0.19 of electricity. I used a borrowed dataset, so only the hardware was under test.

Augment stage: the borrowed dataset's 103 recordings were all made in one room, on one table, under one light. Cosmos3 Transfer, run with the Cosmos3-Nano model, repaints the background but keeps the arm's motion. It ran for six days on the Thor and made 309 new recordings across a kitchen, a lab and a workshop. That first run didn't match the source.

Physical AI flywheel​

This loop comes from the Seeed Studio x NVIDIA Sim-to-Real course and what I've learnt about physical AI on AWS over the past 18 months.

So far I've done one step: training on the Thor. Augment is right on one episode, and the full run is going. A small pilot showed GR00T trains on a mix of real and augmented data. Isaac Sim is an optional second source. The loop hasn't gone round once yet.

Write-ups​

Hardware​

Software​

GR00T N1.7 · Cosmos3 Transfer · LeRobot · PyTorch 2.9 (CUDA 13, aarch64) · torchcodec · JetPack 7.1