Part of ProjectReachy Mini × Strands Agent on Jetson ThorA local-first voice & vision Reachy Mini Lite robot driven by an on-device Strands Agent on NVIDIA Jetson Thor

This is Part 11 of the series. The robot's vision has been local since Part 4, but its brain — the reasoning model the Strands Agent runs on — has been Amazon Bedrock. This post moves the brain on-device too: local Nemotron served by Ollama becomes the default, and Amazon Bedrock is demoted to a single opt-in env var.

With this swap the entire assistant — wake word, vision, reasoning, speech — runs on the Jetson Thor with no network dependency and $0 per call. The design goal is that going local should change nothing else: the same twelve tools, the same system prompt, and the same ModelCallBudget wrap whichever backend is chosen. Only the model object differs.

The two things that make a local reasoning model actually work as an agent brain are (1) it must support function calling — a Strands Agent is useless without tool use — and (2) its chain-of-thought must never be spoken aloud. Both are handled explicitly.

Goals

• Make local Nemotron via Ollama the default agent brain — $0, offline, on the Thor GPU
• Keep Amazon Bedrock Nova 2 Lite as a one-env-var opt-in, behind the same tools and prompt
• Provide an idempotent Ollama bring-up that installs, serves, pulls, and smoke-tests the model
• Assert function-calling support before the agent relies on it
• Strip <think> reasoning from the reply so the robot never speaks its own monologue

The Overall System

One env var (LLM_BACKEND) selects the model. On startup, the local path runs nemotron_setup.sh to guarantee Ollama is up and the model is ready; at run time _build_model() returns either an OllamaModel or a BedrockModel, and everything else in handle_wake is unchanged.

System Components:

1. LLM_BACKEND switch — ollama (default) or bedrock
2. _build_model() — returns the matching Strands model object; the only thing that changes
3. nemotron_setup.sh — idempotent Ollama install/serve/pull/smoke-test bring-up
4. Ollama + Nemotron (nemotron-3-nano:30b) — the local reasoning model on the GPU
5. _clean_reply — strips <think>...</think> before text-to-speech

Interactive Sequence Diagram

Step through startup bring-up and one agent call on the local backend — serve, pull, warm, reason, strip reasoning, speak.

Going Local: Ollama Bring-Up, Then an Agent Call

One env var picks the brain; the tools and prompt are identical either way

0/8

Script

Setup

Ollama

Agent

Model

Speak

Milestone

Complete

8 steps across 6 components • same tools + prompt either backend

One env var swaps the brain — local Nemotron ($0, offline) or Bedrock

Architecture

	Nemotron (default)	Amazon Bedrock Nova 2 Lite
LLM_BACKEND	ollama	bedrock
Model id	nemotron-3-nano:30b	us.amazon.nova-2-lite-v1:0
Location	On-device (Jetson Thor GPU)	AWS cloud
Cost	$0	Per-token
Offline	Yes	No (needs network + AWS creds)
Tools / prompt	Identical	Identical

How it works

The LLM_BACKEND switch

reachy_assistant.py reads one env var to pick the brain. Everything else — the tools, the system prompt, the call budget — is identical either way:

LLM_BACKEND = os.environ.get("LLM_BACKEND", "ollama").lower()   # "ollama" | "bedrock"
OLLAMA_HOST = os.environ.get("OLLAMA_HOST", "http://localhost:11434")
NEMOTRON_MODEL = os.environ.get("NEMOTRON_MODEL", "nemotron-3-nano:30b")

def _build_model():
    if LLM_BACKEND == "bedrock":
        return BedrockModel(model_id=MODEL_ID, region_name=AWS_REGION)
    from strands.models.ollama import OllamaModel
    return OllamaModel(host=OLLAMA_HOST, model_id=NEMOTRON_MODEL)

The default is ollama: a local Nemotron model served at http://localhost:11434, $0 per call and fully offline. LLM_BACKEND=bedrock returns a BedrockModel instead. The _build_model() return value is the only thing that changes — the same tools=[...] list and system_prompt are wired around it.

Bringing up Ollama: nemotron_setup.sh

reachy_assistant.sh runs nemotron_setup.sh on startup whenever LLM_BACKEND=ollama (the default), and aborts with a pointer to LLM_BACKEND=bedrock if it fails. It is the idempotent local-LLM bring-up, running four checks so the model is ready before the agent talks to it:

1. Ollama installed — installs via ollama.com/install.sh if the binary is missing
2. Ollama serving — systemctl start ollama, falling back to nohup ollama serve, then waits on /api/version
3. Model pulled — ollama pull nemotron-3-nano:30b if absent
4. Generation smoke test — a one-line prompt to warm the model into the GPU

Between steps 3 and 4 it asserts the model advertises the tools capability — function calling is required for a Strands Agent, so a model without it is flagged:

if ollama show "${MODEL}" 2>/dev/null | grep -qi "tools"; then
  ok "model advertises 'tools' capability (function calling supported)"
else
  err "model does NOT advertise tool support — agent tool use may fail"
fi

Stripping <think> before speaking

Nemotron is a reasoning model and surfaces its chain of thought inside <think>…</think>. That must not be spoken, so the reply is cleaned before it reaches text-to-speech:

def _clean_reply(text: str) -> str:
    """Strip any <think>...</think> reasoning a reasoning model (Nemotron) may surface."""
    return re.sub(r"(?is)<think>.*?</think>", "", text or "").strip()

Technical Challenges & Solutions

Challenge 1: A swap that changes nothing else

Problem: Moving the brain on-device shouldn't require touching the tools, prompt, budget, or any feature already built around the agent.

Solution: The backend choice is isolated to _build_model(). It returns an OllamaModel or a BedrockModel, and handle_wake wires the identical tools=[...] and system_prompt around whichever it gets — so every other part of the build is backend-agnostic.

Challenge 2: A local model that can't call tools

Problem: A Strands Agent is built on function calling. A local model that doesn't support tool use would load fine and then fail at the first tool call.

Solution: nemotron_setup.sh explicitly checks ollama show for the tools capability during bring-up and flags a model that lacks it — so the requirement is caught at setup, not mid-conversation.

Challenge 3: A flaky local stack at startup

Problem: Ollama might not be installed, not be serving, or not have the model pulled — any of which would make the first wake fail.

Solution: The bring-up is idempotent — install if missing, start if down, pull if absent, then a generation smoke test that also warms the model into the GPU. reachy_assistant.sh runs it before the assistant and aborts cleanly (pointing at the Bedrock fallback) if it can't get the model ready.

Challenge 4: The model speaking its own reasoning

Problem: Reasoning models emit a <think>...</think> block. Spoken aloud, the robot would narrate its internal monologue.

Solution: _clean_reply strips the <think> block with a regex before the text reaches speak, so only the final answer is voiced — the same hygiene applied regardless of backend.

Getting Started

GitHub Repository: https://github.com/chiwaichan/nvidia-jetson-thor-strands-agent-reachy-mini-lite

Run local (default) or Bedrock

./reachy_assistant.sh                       # local Nemotron via Ollama ($0, offline)
LLM_BACKEND=bedrock ./reachy_assistant.sh   # Amazon Nova 2 Lite on Bedrock

The default path runs nemotron_setup.sh to ensure Ollama and Nemotron are ready, then starts the assistant; the Bedrock path skips the Ollama bring-up entirely. Either way, say "Hey Reachy" and the same tools and prompt are in play. A deeper breakdown of every local model lives in the repo's docs/local-models.md.

What's Next

In Part 12 - Robot-State Telemetry to IoT Core, every agent action starts uploading a full snapshot of the robot's state to AWS IoT Core — reusing the same MQTT connection the trigger already holds — so an external system can follow exactly what Reachy is doing in near-real-time.

Summary

This post moved the agent's brain on-device:

• Local by default — LLM_BACKEND=ollama runs Nemotron (nemotron-3-nano:30b) on the Thor GPU, $0 and offline; the whole assistant now runs on-device
• One-var swap — LLM_BACKEND=bedrock returns a BedrockModel (Nova 2 Lite) behind the same tools and prompt; only _build_model() differs
• Idempotent bring-up — nemotron_setup.sh installs, serves, pulls, and smoke-tests Ollama + Nemotron, warming the model into the GPU
• Function-calling guard — bring-up asserts the model advertises the tools capability before the agent relies on it
• Reasoning hygiene — _clean_reply strips <think>...</think> so the robot never speaks its chain of thought