Part of ProjectReachy Mini × Strands Agent on Jetson ThorA local-first voice & vision Reachy Mini Lite robot driven by an on-device Strands Agent on NVIDIA Jetson Thor

This is Part 7 of the series. The robot already wakes to "Hey Reachy" and runs the full per-wake loop. Now I want a second way to trigger it — a message published to AWS IoT Core — so another system (a home-automation rule, a sensor, a script) can ask Reachy to look, move, or react without anyone speaking.

The constraint is that adding a second trigger must not break the first. Voice and MQTT are concurrent sources, but the robot has exactly one owner: both feed a single worker queue, so the motors and the agent are never driven by two sources at once. And the MQTT path is opt-in — with no IoT topic configured, the assistant is voice-only and completely unchanged.

There are no device certificates. The subscription connects over WebSocket + SigV4 using the same AWS credential chain the rest of the assistant already uses, which keeps setup to a couple of environment variables.

Goals

• Add an MQTT trigger alongside the voice wake word, routing published messages into agent tasks
• Connect to AWS IoT Core over WebSocket + SigV4 with the default credential chain — no device certs
• Keep the robot's single-owner guarantee — voice and MQTT enqueue to one worker that runs one task at a time
• Make the MQTT callback fire-and-forget so running an agent never stalls the connection's keep-alive
• Route payloads to the right capability (look / move / emotion / generic)
• Make the whole path opt-in — disabled and invisible unless an IoT topic is configured

The Overall System

A published message flows through IoT Core to the assistant's subscription callback, which routes it and enqueues a request. From there it's indistinguishable from a voice request: the same worker drains the same queue and builds the same fresh per-wake agent.

System Components:

1. send_mqtt.sh (or any publisher) — publishes a JSON message to the IoT topic via the AWS CLI
2. AWS IoT Core — delivers the message to the assistant's QoS 1 subscription
3. on_message callback — decodes, routes via _build_iot_request, and enqueues fire-and-forget
4. _task_q + _worker_loop — the single queue and worker that own the robot
5. handle_wake — builds the fresh per-wake agent, exactly as a voice wake does

Interactive Sequence Diagram

Step through a published look message — delivery, routing, the fire-and-forget enqueue, and the worker running the agent — ending with a state snapshot published back on the same connection.

MQTT as a Second Trigger: Published Message to Agent Task

A fire-and-forget callback routes and enqueues; the same single worker owns the robot

0/9

Publish

IoT Core

Callback

Queue

Worker

Robot

Milestone

Complete

9 steps across 6 components • voice and MQTT share one worker

A published message becomes an agent task — same single-owner robot

Architecture

The listener is gated on configuration and shares the worker with the voice loop:

Env var	Default	Purpose
IOT_ENDPOINT	unset → disabled	ATS data endpoint, e.g. xxxx-ats.iot.us-east-1.amazonaws.com
IOT_TOPIC	unset → disabled (e.g. reachy-mini)	topic to subscribe to
IOT_REGION	us-east-1	SigV4 signing region
IOT_CLIENT_ID	reachy-mini-<pid>	MQTT client id

How it works

Subscribing over WebSocket + SigV4

start_iot_listener connects to AWS IoT Core with mqtt_connection_builder.websockets_with_default_aws_signing from the AWS IoT Device SDK (awsiotsdk/awscrt). There are no device certificates — the connection is signed with SigV4 using AwsCredentialsProvider.new_default_chain(), reusing the same AWS credential chain the rest of the assistant uses:

conn = mqtt_connection_builder.websockets_with_default_aws_signing(
    endpoint=IOT_ENDPOINT,
    region=IOT_REGION,
    credentials_provider=AwsCredentialsProvider.new_default_chain(),
    client_id=IOT_CLIENT_ID,
    clean_session=True,
    keep_alive_secs=30,
)
conn.connect().result()
conn.subscribe(topic=IOT_TOPIC, qos=mqtt.QoS.AT_LEAST_ONCE, callback=on_message)[0].result()

The subscription is QoS 1 (at-least-once). The same live connection is then handed to set_iot_connection so state snapshots can be published back on it (Part 12). The listener only starts when both IOT_ENDPOINT and IOT_TOPIC are set; otherwise it prints a notice and returns None, leaving the assistant voice-only.

One queue, one robot owner

Voice and MQTT are two concurrent trigger sources, but the robot has one owner. Both enqueue onto a single _task_q (queue.Queue) that _worker_loop drains one task at a time, setting _busy while a task runs. The difference is in how each source enqueues:

• The voice loop puts (task, done) with a threading.Event and blocks on done.wait(), so it can reset the mic afterwards.
• The MQTT callback puts (req, None) and returns immediately — fire-and-forget. Running the agent on the MQTT event-loop thread would stall the keep-alive heartbeat, so the callback only routes and enqueues; the worker picks it up.

Because the worker owns the robot, a wake word heard while _busy is set is ignored rather than starting a second concurrent task.

Payload routing

on_message decodes the payload, JSON-parses it (non-JSON passes through as text), and hands it to _build_iot_request, which maps the message to a natural-language request for the agent:

Payload	Route
{"event":"look","question":"..."}	look via the camera (Cosmos Reason 2) and answer
{"event":"move","instruction":"..."}	compose a physical gesture from the motion tools
{"message":"<sentence>"}	read the sentiment and play one matching emotion move
anything else	generic: the agent decides how to react and replies in one short sentence

The event value is matched case-insensitively and accepts aliases (look/describe/vision; move/gesture/motion). The resulting text is enqueued and handled by a fresh Strands Agent in handle_wake, exactly as a voice request would be.

Technical Challenges & Solutions

Challenge 1: A second trigger that doesn't break the first

Problem: Adding MQTT must not regress the voice-only experience or let two sources fight over the robot.

Solution: The MQTT listener is opt-in (disabled unless IOT_ENDPOINT and IOT_TOPIC are set) and enqueues onto the same _task_q the voice loop uses. One worker drains the queue, so there's always a single robot owner; a wake heard mid-task is simply ignored.

Challenge 2: Device-certificate management

Problem: The usual AWS IoT path uses per-device X.509 certificates — provisioning, rotation, and storage to manage on the edge box.

Solution: Connect over WebSocket + SigV4 with AwsCredentialsProvider.new_default_chain(). The assistant reuses the AWS credentials it already has for Bedrock/Lambda/S3 — no certs to provision, and one fewer secret on the device.

Challenge 3: Don't stall the MQTT keep-alive

Problem: If the agent ran synchronously inside the MQTT on_message callback, the long-running task would block the connection's event loop and drop the keep-alive heartbeat, disconnecting the subscription.

Solution: The callback is fire-and-forget — it routes the payload and puts (req, None) onto the queue, then returns immediately. The worker thread runs the agent off the MQTT event loop, so heartbeats keep flowing.

Challenge 4: Heterogeneous payloads

Problem: Publishers send different shapes — a vision question, a motion instruction, a sentiment sentence, or some arbitrary event.

Solution: _build_iot_request maps known event values (with aliases) to the matching capability and falls back to a generic "decide how to react" request for anything else — so even an unmodelled {"event":"door_open"} produces a sensible reaction.

Getting Started

GitHub Repository: https://github.com/chiwaichan/nvidia-jetson-thor-strands-agent-reachy-mini-lite

Publish a trigger

With the assistant running (./reachy_assistant.sh with an IoT topic configured), publish with the bundled helper — it uses the AWS CLI and resolves the account's ATS endpoint automatically:

./send_mqtt.sh '{"event":"look","question":"what color is the wall?"}'
./send_mqtt.sh '{"message":"great news, you passed the test"}'
./send_mqtt.sh '{"event":"move","instruction":"nod twice, then look around"}'
./send_mqtt.sh                       # default sample payload

Each message is delivered to the subscription, routed, enqueued, and handled by a fresh agent — exactly as a spoken request would be.

What's Next

In Part 8 - Emotion Moves, I give the agent ~80 pre-choreographed expressive moves and let it pick one to match the sentiment of a request or a published message — so "great news, you passed the test" plays a proud or happy animation.

Summary

This post covered the MQTT trigger — a second, opt-in wake source:

• WebSocket + SigV4 — start_iot_listener connects with the default AWS credential chain, no device certificates, QoS 1
• Single robot owner — voice and MQTT both enqueue to one _task_q drained by one worker, so two sources never drive the motors at once
• Fire-and-forget callback — on_message routes and enqueues (req, None) and returns immediately, so the agent never stalls the MQTT keep-alive
• Payload routing — _build_iot_request maps look / move / message (and aliases) to the right capability, with a generic fallback for anything else
• Opt-in and invisible — disabled unless IOT_ENDPOINT and IOT_TOPIC are set, so a voice-only run is unchanged