2 posts tagged with "rvr"

These are the slides from my talk at the AWS User Group Wellington meet-up on 26 May 2026, walking through how I built a real-time pipeline that listens to spoken commands in the browser and drives a Sphero RVR — exact distances, exact angles — with live telemetry streaming back into the UI.

Tip: click into the slides and use the arrow keys to navigate, or hit the fullscreen button for the best experience.

What the talk covers​

• The problem — turning a spoken sentence into a precise, exact-distance robot motion in real time
• Voice in — Amazon Nova 2 Sonic bidirectional streaming with forced tool use as a "robot controller"
• Commands down — browser → AWS IoT Core MQTT publish via temporary Cognito credentials, no backend
• The edge — a Seeed Studio XIAO ESP32-S3 bridging WiFi/MQTT to the Sphero RVR's UART SDK
• Maneuver framework — locator-based distance and IMU-based turn, both running on the device, so "1 metre" actually means 1 metre
• Telemetry up — an AWS CDK stack tying IoT Rule → Lambda → AppSync → DynamoDB → live GraphQL subscription back into the React UI

The three repos behind the demo​

• arduino-aws-iot — ESP32-S3 firmware. The talk focuses on the sphero_rvr device group, including the new maneuver executor (forward, reverse, turn, cancel) layered on top of the existing drive D-pad.
• cdk-iot-sphero-rvr-streaming — AWS CDK stack that filters the RVR telemetry MQTT topic, flattens the nested payload in Lambda, and forwards it as a GraphQL mutation to an Amplify-managed AppSync API.
• amplify-react-nova-sonic-voice-chat-sphero-rvr — React + AWS Amplify Gen 2 frontend. The same Cognito identity is used for Bedrock streaming, IoT publish, and the AppSync subscription that surfaces live telemetry.

Related deep-dive posts​

If you want background on the building blocks before the talk:

1. Sphero RVR with AWS IoT Core on a Seeed Studio XIAO ESP32-S3
2. Real-Time Voice Chat with Amazon Nova Sonic, React and AWS Amplify Gen 2

Thanks​

Big thanks to the AWS User Group Wellington organisers and everyone who came along — happy to chat about any of the code, the maneuver framework, or where the project goes next.

A Sphero RVR integrated with a Seeed Studio XIAO ESP32S3 with telemetry uploaded into, and also, basic drive remote control commands received from any where leveraging AWS IoT Core.

Overview​

Lately I have been aiming to go deep on AI Robotics, and last year I have been slowly experimenting more and more with anything that is AI, IoT and Robotics related; with the intention of learning and going as wide and as deep as possible in any pillars I can think of. You can check out my blogs under the Robotics Project to see what I have been up to. This year I want to focus on enabling mobility for my experiments - as in providing wheels for solutions to move around the house, ideally autonomously; starting off with wheel based solutions bought off-shelve, followed by solutions that I build myself from open-sourced projects people have kindly contirbuted online, and then ambitiously designed, 3D Printed and built all from the ground up - perhaps in a couple of years time.

This project uses a Seeed Studio XIAO ESP32S3 microcontroller to communicate with a Sphero RVR robot via UART, while simultaneously connecting to AWS IoT Core over WiFi. The system publishes real-time sensor telemetry and accepts remote drive commands through MQTT.

Hardware Components​

Component	Description
Seeed Studio XIAO ESP32S3	Compact ESP32-S3 microcontroller with WiFi, 8MB flash
Sphero RVR	Programmable robot with motors, IMU, color sensor, encoders
XIAO Expansion Board	Provides OLED display (128x64 SSD1306) for status info

Hardware Wiring​

Features​

Real-time Telemetry​

The system publishes comprehensive sensor data every 60 seconds:

• IMU Data: Pitch, roll, yaw orientation
• Accelerometer & Gyroscope: Motion and rotation data
• Color Sensor: RGB values with confidence
• Compass: Heading in degrees
• Ambient Light: Lux measurements
• Motor Thermal: Temperature and protection status
• Encoders: Wheel tick counts
• Position & Velocity: Locator data in meters

Remote Commands via MQTT​

Control the RVR from anywhere using JSON commands:

• Drive: Speed and heading control
• Tank: Independent left/right motor control
• Raw Motors: Direct motor speed control
• LED Control: Headlights, brakelights, status LEDs
• Navigation: Reset yaw, reset locator
• Power: Wake and sleep commands

Local OLED Display​

The XIAO Expansion Board's OLED display shows real-time sensor readings for local monitoring.

MQTT Message Flow​

Sensor Data Pipeline​

Architecture​

The XIAO ESP32S3 acts as a bridge between the Sphero RVR and AWS IoT Core:

1. UART Communication: The ESP32S3 communicates with the RVR via UART (GPIO43/44)
2. WiFi Connection: Connects to local WiFi network
3. MQTT over TLS: Secure connection to AWS IoT Core with X.509 certificates
4. Bidirectional: Publishes telemetry and subscribes to command topics

Sphero RVR Protocol​

The Sphero RVR uses a binary packet-based protocol over UART. Each packet contains a start-of-packet byte (0x8D), an 8-byte header with device ID and command ID, variable-length data body, checksum, and end-of-packet byte (0xD8). The RVR has two internal processors: Nordic (handles BLE, power, color detection) and ST (handles motors, IMU, encoders).

Source Code​

I ported the code into this project to control the RVR using the UART protocol based on the Sphero SDK.

You can find the source code for this project here: https://github.com/chiwaichan/platformio-aws-iot-seeed-studio-esp32s3-sphero-rvr