15 posts tagged with "edge-ai"

This is Part 15, the final post in the series. Throughout the build, every wake has created a brand-new agent and destroyed it — which keeps idle at $0 but means the robot forgets everything the instant it replies. This post fixes that without giving up the cost-minimal lifecycle: the robot gains conversational memory across wakes, so you can ask a follow-up — "…and what was the first thing I said?" — and the fresh per-wake agent recalls it. It also rounds out the agent's body with six composable motion tools it can chain into novel gestures.

This is Part 14 of the series. Several features now want the camera — face tracking, the idle watcher, Cosmos vision, clip recording — and the voice loop wants the mic. But /dev/video0 and the ALSA mic are single-opener devices: once one process holds either, no one else can. Until now everything that needed a device had to live as a thread inside the one process that opened it.

This is Part 13 of the series. The telemetry from Part 12 tells a subscriber what the robot did — but not what it saw. This post adds that: each interaction (and each presence detection) is recorded to a short MP4, uploaded to Amazon S3, and a presigned download URL is attached to the outgoing message, so whoever receives the event can watch the exact clip.

This is Part 12 of the series. The robot now does a lot — moves, looks, emotes, answers, notices presence. I want an external system (a dashboard, a data lake, another robot) to be able to follow exactly what Reachy is doing in near-real-time. So every agent action uploads a full snapshot of the robot's state — servos, head pose, daemon status, runtime flags — to AWS IoT Core.

This is Part 11 of the series. The robot's vision has been local since Part 4, but its brain — the reasoning model the Strands Agent runs on — has been Amazon Bedrock. This post moves the brain on-device too: local Nemotron served by Ollama becomes the default, and Amazon Bedrock is demoted to a single opt-in env var.

This is Part 10 of the series. The robot can already look on demand with Cosmos vision and notice who's around while idle. Now I want it to feel present: its head should follow your face in real time — and it has to do that while Cosmos vision and the idle watcher are also using the camera.

This is Part 9 of the series. So far the robot only acts when triggered — a wake word or an MQTT message. Now I want it to show a little life between interactions: while resting, it should quietly notice who's in the room and have a place to hang future reactions — greet a person, perk up at a cat.

This is Part 8 of the series. The robot can move with primitive gestures, but those are deliberate, literal motions. For expression — reacting to good news, bad news, a greeting, a joke — I want something richer: a library of ~80 pre-choreographed emotion moves that the agent picks from to match the mood.

This is Part 7 of the series. The robot already wakes to "Hey Reachy" and runs the full per-wake loop. Now I want a second way to trigger it — a message published to AWS IoT Core — so another system (a home-automation rule, a sensor, a script) can ask Reachy to look, move, or react without anyone speaking.

This is Part 6 of the series. So far the robot can move, see, and run the full per-wake loop. Now I want it to answer questions about real data — "Hey Reachy, has the kitchen water sensor tripped today?" — by querying an AWS data lake of IoT sensor readings, and speaking the answer back in one sentence.

This is Part 5 of the series — and the centerpiece. Part 3 gave the robot ears and Part 4 gave it eyes. This post is the loop that ties them — and everything still to come — together: "Hey Reachy" → transcribe → a fresh per-wake Strands Agent that picks a tool → speak one short sentence → tear down.

This is Part 4 of the series. In Part 3 the robot learned to listen. Now I want to give it eyes: the ability to look through its camera and answer a question about the room — identify an object, read visible text, count things, or describe what a person is doing — entirely on-device.

This is Part 3 of the series. In Part 2 I proved the hardware works — the head moves, the mic hears, the camera sees. Now the robot can start listening. This post covers the lowest layer of the voice stack: the "Hey Reachy" wake word, running fully offline with no LLM and no network.

This is Part 2 of the series building a local-first voice & vision robot on an NVIDIA Jetson Thor. In Part 1 I covered the motion foundation — a Strands Agent given motion tools that drive the robot. But all of that assumes one thing: that the host can actually reach the robot, and that the hardware underneath works. This post is about earning that assumption.

This is Part 1 of a series that builds a Reachy Mini Lite desk robot driven by a Strands agent running entirely on-device on an NVIDIA Jetson Thor. Say "Hey Reachy" (or publish an MQTT message) and a fresh agent wakes up, decides which tool it needs — move its head and body, see the room with a local vision model, express an emotion, or answer questions about IoT sensor data in an AWS data lake — speaks one short sentence, and tears itself down. Idle is pure-local and $0: no cloud, no LLM tokens, just an offline wake-word listener.