Part of ProjectReachy Mini × Strands Agent on Jetson ThorA local-first voice & vision Reachy Mini Lite robot driven by an on-device Strands Agent on NVIDIA Jetson Thor

This is Part 15, the final post in the series. Throughout the build, every wake has created a brand-new agent and destroyed it — which keeps idle at $0 but means the robot forgets everything the instant it replies. This post fixes that without giving up the cost-minimal lifecycle: the robot gains conversational memory across wakes, so you can ask a follow-up — "…and what was the first thing I said?" — and the fresh per-wake agent recalls it. It also rounds out the agent's body with six composable motion tools it can chain into novel gestures.

The trick is that memory is bridged on disk, not in the process. Each wake still builds a fresh agent and dels it; what persists is a small local JSON session store. The next wake's agent, pointed at the same conversation id, reloads the recent turns. It's built entirely on Strands' own session managers — no database, no cloud — and the store lives in a durable cache path, so memory survives a reboot. Idle stays $0 because sessions are just message storage; no LLM runs until a wake.

Goals

• Let a fresh per-wake agent recall recent turns across separate wakes (voice and MQTT)
• Keep the cost-minimal lifecycle — still a new agent per wake, still $0 idle
• Persist turns as local JSON via Strands' FileSessionManager — no DB, no cloud, reboot-safe
• Bound replayed context with a sliding window so tokens and latency don't grow unbounded
• Rotate to a new conversation after a period of inactivity
• Let the agent compose six motion primitives into novel gestures on both voice and MQTT

The Overall System

A rotating conversation id ties successive wakes into one thread. When memory is on, handle_wake attaches a FileSessionManager (persisting each turn to JSON) and a SlidingWindowConversationManager (bounding replay). The next wake's fresh agent, pointed at the same id, reloads the recent turns before reasoning.

System Components:

1. _session_for_now() — returns the active conversation id, rotating after SESSION_TTL idle
2. FileSessionManager — persists every turn to JSON under SESSION_DIR
3. SlidingWindowConversationManager — bounds replayed history to SESSION_WINDOW messages
4. _touch_interaction() — refreshes the idle clock at end-of-task so long tasks don't rotate early
5. Six motion tools — nod, shake_head, look_around, wiggle_antennas, spin_body, move_head

Interactive Sequence Diagram

Step through two wakes — persist and tear down on the first, reload and recall on the second — and see how memory survives the agent's destruction.

Memory Across Two Wakes — Bridged on Disk

Each wake builds a fresh agent and tears it down, yet recalls prior turns from local JSON

0/9

Wake

Session

Agent

Disk

Window

Milestone

Complete

9 steps across 5 components • local JSON sessions, reboot-safe, $0 idle

A fresh agent per wake — yet it remembers, by bridging memory on disk

Architecture

Env var	Default	Meaning
SESSION_MEMORY	1	0 restores stateless-per-wake
SESSION_DIR	~/.cache/reachy_voice/sessions	durable JSON store, reboot-safe
SESSION_TTL	300	idle seconds before the id rotates to a new conversation
SESSION_WINDOW	40	max messages replayed into a wake

How it works

Memory across wakes

Every wake builds a fresh agent and dels it afterwards (del agent; gc.collect()), so in-memory history dies with it. Memory is bridged on disk instead. In handle_wake, when a conversation id is active, the agent gets a FileSessionManager and a SlidingWindowConversationManager:

session_id = _session_for_now()
if session_id is not None:
    agent_kwargs.update(
        agent_id="reachy",
        session_manager=FileSessionManager(session_id=session_id, storage_dir=SESSION_DIR),
        conversation_manager=SlidingWindowConversationManager(window_size=SESSION_WINDOW),
    )
agent = Agent(**agent_kwargs)

FileSessionManager persists each turn to local JSON under SESSION_DIR — no DB, no cloud — at <storage_dir>/session_<id>/agents/agent_<id>/messages/message_<n>.json. The next wake's fresh agent, pointed at the same (session_id, agent_id), reloads those messages. The default SESSION_DIR (~/.cache/reachy_voice/sessions) is a durable path, so memory survives a reboot. Sessions are just message storage, so no LLM runs while idle — memory stays $0. SlidingWindowConversationManager bounds how much history is replayed into each wake (SESSION_WINDOW, default 40), capping input tokens and latency — SlidingWindow is used rather than Summarizing, which would call the model.

One rotating conversation id across wakes

All wakes — voice and MQTT — share a single conversation thread so the robot remembers recent context. _session_for_now() returns the active id, rotating to a fresh one after SESSION_TTL seconds of inactivity:

def _session_for_now() -> str | None:
    global _session_id, _session_seq, _last_interaction
    if not SESSION_MEMORY:
        return None
    now = time.time()
    if _session_id is None or (now - _last_interaction) > SESSION_TTL:
        _session_seq += 1
        _session_id = time.strftime("conv-%Y%m%d-%H%M%S-") + str(_session_seq)
    _last_interaction = now
    return _session_id

The idle gap is measured end-to-start: _touch_interaction() refreshes the clock at the end of each task so a long task doesn't trigger a premature rotation on the next wake. When SESSION_MEMORY=0, _session_for_now() returns None, the managers aren't attached, and behaviour reverts to stateless-per-wake exactly as before.

Six composable motion tools

Six primitive motion tools are given to the per-wake agent so it can compose gestures beyond the canned play_emotion clips. Each wraps the Reachy Mini Python SDK (goto_target / create_head_pose), clamps its inputs, returns to neutral where it makes sense, publishes a motion state snapshot, and returns a short status string:

Tool	Drives	Clamp
nod(times)	head pitch up/down	times 1–5
shake_head(times)	head yaw left/right	times 1–5
look_around()	head yaw sweep left→right→center	—
wiggle_antennas(times)	both antennas	times 1–6
spin_body(degrees, duration)	body yaw	degrees −160..160
move_head(pitch, roll, yaw, duration)	absolute head orientation	pitch/roll −40..40, yaw −180..180

They're safe to call mid-task: the face tracker is paused while _busy is set, and handle_wake recenters the head afterwards. A gesture is a first-class trigger on both voice and the MQTT {"event":"move","instruction":"..."} route — both feed _move_request(), which wraps the instruction so the agent performs every step in order, one tool per step, before replying. (The motion foundation is covered in depth in Part 1.)

Technical Challenges & Solutions

Challenge 1: Memory without giving up the $0 lifecycle

Problem: The whole build relies on destroying the agent after every wake — but that's exactly what erases conversation history.

Solution: Bridge memory on disk, not in the process. FileSessionManager persists each turn to local JSON; the next fresh agent reloads it by (session_id, agent_id). The agent is still built and destroyed per wake, and since sessions are just stored messages, idle remains $0 — no LLM runs until a wake.

Challenge 2: Unbounded context growth

Problem: Replaying an ever-growing history into each wake would steadily inflate input tokens and latency.

Solution: A SlidingWindowConversationManager caps replay at SESSION_WINDOW (default 40) messages — older turns drop off. SlidingWindow is chosen over Summarizing precisely because summarising would call the model and break the $0-idle guarantee.

Challenge 3: When does one conversation end and another begin?

Problem: Without rotation, every interaction forever would accrete into a single thread; rotate too eagerly and a brief pause loses context mid-conversation.

Solution: _session_for_now() rotates the conversation id after SESSION_TTL (default 300 s) of inactivity, and _touch_interaction() measures that gap end-of-task → next wake, so a long task doesn't cause a premature rotation. A natural pause starts a fresh conversation; a quick follow-up continues the current one.

Challenge 4: Surviving a reboot

Problem: In-memory or temp-dir memory would vanish on restart.

Solution: SESSION_DIR defaults to a durable cache path (~/.cache/reachy_voice/sessions), so the JSON session store persists across reboots — the robot can recall a conversation from before it was power-cycled.

Getting Started

GitHub Repository: https://github.com/chiwaichan/nvidia-jetson-thor-strands-agent-reachy-mini-lite

Try memory and composed gestures

./reachy_assistant.sh                       # memory on by default
SESSION_MEMORY=0 ./reachy_assistant.sh      # stateless per wake (old behaviour)
SESSION_TTL=60 ./reachy_assistant.sh        # rotate to a new conversation after 60s idle

# compose a multi-step gesture over MQTT
./send_mqtt.sh '{"event":"move","instruction":"nod twice, then look around the room"}'

Say "Hey Reachy", ask something, then in a later wake ask a follow-up that refers back — the fresh agent recalls it. Pause longer than SESSION_TTL and the next wake starts a new conversation.

Wrapping up the series

That completes the build: a Reachy Mini Lite desk robot driven by a Strands Agent running entirely on a Jetson Thor — it moves, sees, listens, emotes, answers questions about real data in S3 Tables, follows your face, streams its state and clips to the cloud, and now remembers. Idle is pure-local and $0; the cloud is everywhere optional. The full code — every feature in this series — is in the project repository.

Summary

This final post added memory and rounded out motion:

• Memory across wakes — a fresh per-wake agent recalls recent turns via Strands' FileSessionManager, persisted as local JSON, no DB or cloud
• Still $0 idle — sessions are just message storage; no LLM runs until a wake, and the agent is still built and destroyed per wake
• Bounded + reboot-safe — SlidingWindowConversationManager caps replay at SESSION_WINDOW, and a durable SESSION_DIR survives restarts
• Conversation rotation — _session_for_now() starts a new conversation after SESSION_TTL idle, measured end-of-task → next wake
• Composable motion — six clamped, return-to-neutral motion tools let the agent build novel multi-step gestures on both voice and MQTT