Skip to main content

4 posts tagged with "Python"

Python

View All Tags

FeedMyFurBabies – Send and Receive MQTT messages between AWS IoT Core and your micro-controller – I am switching from Arduino CPP to MicroPython

· 7 min read
Chiwai Chan
Tinkerer

Recently I switched my Cat Feeder project's IaC to AWS CDK in favour of increasing my focus and productivity on building and iterating, rather than constantly mucking around with infrastructure everytime I resume my project after a break; which is rare and far between these days.

Just as with coding IoT microcontrollers such as the ESP32s, I want to get straight back into building every opportunity I get; so I am also switching away from Arduino based microcontroller development written in C++ - I don't have a background in C++ and to be honest this is the aspect I struggled with the most because I tend to forget things after not touching it for 6 months or so.

So I am switching to MicroPython to develop the logic for all my IoT devices going forward, this means I get to use Python - a programming lanaguge I work with frequently so there is less chance of me being forgetful when I use it at least once a month. MicroPython is a lean and efficient implementation of the Python 3 programming language that includes a subset of the Python standard library and is optimized to run on microcontrollers and in constrained environments - a good fit for IoT devices such as the ESP32!

What about all the Arduino hardware and components I already invested in?

Good news is MircoPython is supported on all ESP32 devices - based on the ones I myself have purchased; all I need to do to each ESP32 device is to flash it with a firmware - if you are impatient, you can scroll down and skip to below to the flashing the firmware section. When I first started Arduino, MicroPython was available to use, but that was 2 years ago and there were not as many good blog and tutorial content out there as there is today; I couldn't at the time work out how to control components such as sensors, servos and motors as well as I could with C++ based coding using Arduino; nowdays there are way more content to learn off and I've learnt (by PoCing individual components) enough to switch to MicroPython. As far as I understand it, any components you have for Arduino can be used in MicroPython, provided that there is a library out there that supports it, if there isn't then you can always write your own!

What's covered in this blog?

By the end of this blog, you will be able to send and receive MQTT messages from AWS IoT core using MicroPython, I will also cover the steps involved in flashing a MicroPython firmware image onto an ESP32C3. Although this blog has a focus and example on using an ESP32, this example can be applied to any micro-controllers of any brand or flavours, provided the micro-controller you are using supports MicroPython.

Flashing the MicroPython firmware onto a Seeed Studio XIAO ESP32C3

Seeed Studio XIAO ESP32C3

The following instructions works for any generic ESP32C3 devices!

Download the latest firmware from micropython.org

https://micropython.org/download/ESP32_GENERIC_C3/

MicroPython firmware esp32c3

Next, I connected my ESP32C3 to my Mac and ran the following command to find the name of the device port

 /dev/ttyUSB0

Find port device

My ESP32C3 is named "/dev/tty.usbmodem142401", the name for your ESP32C3 may be different.

Next, install esptool onto your computer, then run the following commands to flash the MicroPython firmware onto the ESP32C3 using the bin file you've just downloaded.

esptool.py --chip esp32c3 --port /dev/tty.usbmodem142401 erase_flash

esptool.py --chip esp32c3 --port /dev/tty.usbmodem142401 --baud 460800 write_flash -z 0x0 ESP32_GENERIC_C3-20240105-v1.22.1.bin

It should look something like this when you run the commands.

esptool Flashing Firmware

Install Thonny and run it. Then go to Tools -> Options, to configure the ESP32C3 device in Thonny to match the settings shown in the screenshot below.

esptool Flashing Firmware

If everything went well, you should see these 2 sections in Thonny: "MicroPython Device" and "Shell", if not then try clicking on the Stop button in the top menu.

Thonny MicroPython Device

AWS IoT Core Certificates and Keys

In order to send MQTT messages to an AWS IoT Core Topic, or to receive a message from a Topic in reverse, you will need a set of Certificate and Key\s for your micro-controller; as well as the AWS IoT Endpoint specific to your AWS Account and Region.

It's great if you have those with you so you can skip to the next section, if not, do not worry I've got you covered. In a past blog I have a reference architecture accompanied by a GitHub repository on how to deploy resources for an AWS IoT Core solution using AWS CDK, follow that blog to the end and you will have a set of Certificate and Key to use for this MicroPython example; the CDK Stack will deploy all the neccessary resources and policies in AWS IoT Core to enable you to both send and receive MQTT messages to two separate IoT Topics.

Reference AWS IoT Core Architecture: https://chiwaichan.co.nz/blog/2024/02/02/feedmyfurbabies-i-am-switching-to-aws-cdk/

Upload the MicroPython Code to your device

Now lets upload the MicroPython code to your micro-controller and prepare the IoT Certificate and Key so we can use it to authenticate the micro-controller to enable it to send and receive MQTT messages between your micro-controller and IoT Core.

Clone my GitHub repository that contains the MicroPython example code to publish and receive MQTT message with AWS IoT Core: https://github.com/chiwaichan/feedmyfurbabies-micropython-iot

It should look something like this.

GitHub repo

Copy your Certificate and Key into the respective files shown in the above screenshot; otherwise, if you are using the Certificate and Key from my reference architecture, then you should use the 2 Systems Manager Parameter Store values create by the CDK Stack.

Systems Manager Parameter Store values

Next we convert the Certificate and Key to DER format - converting the files to DER format turns it into a binary format and makes the files more compact, especially neccessary when we run use it on small devices like the ESP32s.

In a terminal go to the certs directory and run the following commands to convert the certificate.pem and private.key files into DER format.

openssl rsa -in private.key -out key.der -outform DER
openssl x509 -in certificate.pem -out cert.der -outform DER

You should see two new files with the DER extension appear in the directory if all goes well; if not, you probably need to install openssl.

Systems Manager Parameter Store values

In Thonny, in the Files explorer, navigate to the GitHub repository's Root directory and open the main.py file. Fill in the values for the variables shown in the screenshot below to match your environment, if you are using my AWS CDK IoT referenece architecture then you are only required to fill in the WIFI details and the AWS IoT Endpoint specific to your AWS Account and Region.

Wifi and iot Core Settings

Select both the certs folder and main.py in the Files explorer, then right click and select "Upload to /" to upload the code to your micro-controller; the files will appear in the "MicroPython Device" file explorer.

Upload files to Thonny

This is the moment we've been waiting for, lets run the main.py Python script by clicking on the Play Icon in green.

Run main

If all goes well you should see some output in the Shell section of Thonny.

Thonny Shell

The code in the main.py file has a piece of code that is generating a random number for the food_capacity percentage property in the MQTT message; you can customise the message to fit your use case.

But lets verify it is actually received by AWS IoT Core.

aws iot mqtt test client

Alright, lets go the other way and see if we can receive MQTT messages from AWS IoT Core using the other Topic called "cat-feeder/action" we subscribed to in the MicroPython code.

Lets go back the AWS Console and use the MQTT test client to publish a message.

publlish mqtt from aws

thonny message received

In the Thonny Shell we can see the message "Hello from AWS IoT console" sent from the AWS IoT Core side and it being received by the micro-controller.

Smart Cat Feeder – Part 4

· 5 min read
Chiwai Chan
Tinkerer

This is the Part 4 and final blog of the series where I detail my journey in learning to build an IoT solution.

Please have a read of my previous blogs to get the full context leading up to this point before continuing.

  • Part 1: I talked about setting up a Seeed AWS IoT Button
  • Part 2: I talked about publishing events to an Adruino Micro-controller from AWS
  • Part 3: I talked about my experience of using a 3D Printer for the first time to print a Cat Feeder

Why am I building this Feeder?

I've always wanted to dip my toes into building IoT solutions beyond doing what a typical tutorial teaches in only turning on LEDs - I wanted to build something that would used everyday. Plus, I often forget to feed the cats while I am away from home (for the day), so it would be nice to come home to a non-grumpy cat by feeding them remotely any time and from any where in the world using the internet.

What was used to build this Feeder?

  • A 3D Printer using PLA as the filament material.
  • An Arduino based micro-controller - in this case a Seeed Studio XIAO ESP32C3
  • A couple of motors and controllers
  • AWS Services
  • Seeed AWS IoT Button
  • Some code
  • and some cat food

So how does it work and how is it put together?

To simply describe what is built, the Feeder uses an Iot button click to trigger events over the internet to instruct the feeder to dispense food into one or both food bowls.

cat feeder

Here are some diagrams describing the architecture of the solution - the technical things that happens in-between the IoT button and the Cat Feeder.

architecture diagram seeed sequence diagram

When the Feeder receives a MQTT message from the AWS IoT Core Service, it runs the motor for 10 seconds to dispense food into either one of food bowls, and if the message contains an event value to dispense food into both bowls we can run both motors concurrently using the L298N controller.

Here's a video of some timelapse picture captured during the 3 weeks it took to 3D print the feeder.

The Feeder is made up of a small handful of basic hardware components, below is a Breadboard diagram depicting the components used and how they are all wired up together. A regular 12V 2A DC power adapter supply is used to power all the components.

breadboard diagram seeed

The code to start and stop a motor is about 10 lines of code as shown below. This is the completed version of the Arduino Sketch shown in Part 2 of this blog series when it was partially written at the time.

#include "secrets.h"
#include <WiFiClientSecure.h>
#include <MQTTClient.h>
#include <ArduinoJson.h>
#include "WiFi.h"

// The MQTT topics that this device should publish/subscribe
#define AWS_IOT_PUBLISH_TOPIC "cat-feeder/states"
#define AWS_IOT_SUBSCRIBE_TOPIC "cat-feeder/action"

WiFiClientSecure net = WiFiClientSecure();
MQTTClient client = MQTTClient(256);

int motor1pin1 = 32;
int motor1pin2 = 33;
int motor2pin1 = 16;
int motor2pin2 = 17;

void connectAWS()
{
WiFi.mode(WIFI_STA);
WiFi.begin(WIFI_SSID, WIFI_PASSWORD);

Serial.println("Connecting to Wi-Fi");
Serial.println(AWS_IOT_ENDPOINT);

while (WiFi.status() != WL_CONNECTED) {
delay(500);
Serial.print(".");
}

// Configure WiFiClientSecure to use the AWS IoT device credentials
net.setCACert(AWS_CERT_CA);
net.setCertificate(AWS_CERT_CRT);
net.setPrivateKey(AWS_CERT_PRIVATE);

// Connect to the MQTT broker on the AWS endpoint we defined earlier
client.begin(AWS_IOT_ENDPOINT, 8883, net);

// Create a message handler
client.onMessage(messageHandler);

Serial.println("Connecting to AWS IOT");
Serial.println(THINGNAME);

while (!client.connect(THINGNAME)) {
Serial.print(".");
delay(100);
}

if (!client.connected()) {
Serial.println("AWS IoT Timeout!");
return;
}

Serial.println("About to subscribe");
// Subscribe to a topic
client.subscribe(AWS_IOT_SUBSCRIBE_TOPIC);

Serial.println("AWS IoT Connected!");
}

void publishMessage()
{
StaticJsonDocument<200> doc;
doc["time"] = millis();
doc["state_1"] = millis();
doc["state_2"] = 2 * millis();
char jsonBuffer[512];
serializeJson(doc, jsonBuffer); // print to client

client.publish(AWS_IOT_PUBLISH_TOPIC, jsonBuffer);

Serial.println("publishMessage states to AWS IoT" );
}

void messageHandler(String &topic, String &payload) {
Serial.println("incoming: " + topic + " - " + payload);

StaticJsonDocument<200> doc;
deserializeJson(doc, payload);
const char* event = doc["event"];

Serial.println(event);

feedMe(event);
}

void setup() {
Serial.begin(9600);
connectAWS();

pinMode(motor1pin1, OUTPUT);
pinMode(motor1pin2, OUTPUT);
pinMode(motor2pin1, OUTPUT);
pinMode(motor2pin2, OUTPUT);
}

void feedMe(String event) {
Serial.println(event);

bool feedLeft = false;
bool feedRight = false;

if (event == "SINGLE") {
feedLeft = true;
}
if (event == "DOUBLE") {
feedRight = true;
}
if (event == "LONG") {
feedLeft = true;
feedRight = true;
}

if (feedLeft) {
Serial.println("run left");
digitalWrite(motor1pin1, HIGH);
digitalWrite(motor1pin2, LOW);
}

if (feedRight) {
Serial.println("run right");
digitalWrite(motor2pin1, HIGH);
digitalWrite(motor2pin2, LOW);
}

delay(10000);
digitalWrite(motor1pin1, LOW);
digitalWrite(motor1pin2, LOW);
digitalWrite(motor2pin1, LOW);
digitalWrite(motor2pin2, LOW);
delay(2000);

Serial.println("fed");
}

void loop() {
publishMessage();
client.loop();
delay(3000);
}

Demo Time

The Seeed AWS IoT Button is able to detect 3 different types of click events: Long, Single and Double, and we are able to leverage this all the way to the feeder so we will have it performing certains actions base on the click event type.

The video below demonstrates the following scenarios:

  • Long Click: this will dispense food into both cat bowls
  • Single Click: this will dispense food into Ebok's cat bowl
  • Double Click: this will dispense food into Queenie's cat bowl

What's next?

Build the nervous system of an ultimate nerd project I have in mind that would allow me to voice control actions controlling servos, LEDs and audio outputs, by using a mesh of Seeed XIAO BLE Sense micro-controllers and TinyML Machine Learning.

Smart Cat Feeder – Part 2

· 8 min read
Chiwai Chan
Tinkerer

seeed studio xiao esp32c3

The source code for this blog can be found in my Github repository: https://github.com/chiwaichan/aws-iot-cat-feeder. This repository only includes the source code for the solution implemented up to this stage/blog in the project.

In the end I decided to go with the Seeed Studio XIAO ESP32C3 implementation of the ESP32 micro-controller for $4.99 (USD). I also ordered some other bits and pieces from AliExpress that's going to take some time to arrive.

In this Part 2 of the blog series I will demonstrate the exchange of messages (JSON payload) using the MQTT protocol between the ESP32 and the AWS IoT Core Service, as well as the exchange of messages between a Lambda Function and the ESP32 - this Lambda is written in Python which is intended to replace the Lambda triggered by the IoT button event found in Part 1.

Prerequisites if you like to try out the solution using the source code

  • An AWS account.
  • An IoT button. Follow Part 1 of this blog series to onboard your IoT button into the AWS IoT 1-Click Service.
  • Create 2 Certificates in the AWS IoT Core Service. One certificate is for the ESP32 to publish and subscribe to Topics to IoT Core, and the other is used by the IoT button's Lambda to publish a message to a Topic subscribed by the ESP32.

aws iot certificate list

Create a Certificate using the recommended One-Click option.

aws iot certificate create

Download the following files and take note of which device (the ESP32 or the IoT Lambda) you like to use this certificate for:

aws iot certificate created

Activate the Certificate.

aws iot certificate activated

Click on Done. Then repeat the steps to create the second Certificate.

Publish ESP32 States to AWS IoT Core

seeed studio xiao esp32c3 aws iot

The diagram above depicts the components used that is required in order for the ESP32 to send the States of the Cat Feeder, I've yet to decide what to send but examples could be 1.) battery level 2.) Cat weight (based on a Cat's RFID chip and some how weighing them while they eat) 3.) or how much food is remaining in the feeder. So many options.

  1. ESP32: This is the micro-controller that will eventually have a bunch of hardware components that we will take States from, then publish to a Topic.
  2. MQTT: This is the lightweight pub/sub protocol used to send IoT messages over TCP/IP to AWS IoT Core.
  3. AWS IoT Core: This is the service that will forward message to the ESP32 micro-controller that are subscribed to Topics.
  4. IoT Topic: The Lambda will publish a message along with the type of button event (One click, long click or double click) to the Topic "cat-feeder/action", the value of the event is subject to what is supported by the IoT button you use.
  5. Do something later on: I'll decide later on what to do downstream with the State values. This could be anything really, e.g. save a time series of the data into a database or bunch of DynamoDB tables, or get an alert to remind me to charge the Cat Feeder's battery with a customizable threshold?

Instructions to try out the Arduino/ESP32 part of the solution for yourself

  1. Install the Arduino IDE.
  2. Follow this AWS blog on setting up an IoT device, start from "Installing and configuring the Arduino IDE" to including "Configuring and flashing an ESP32 IoT device". Their blog walks us through on preparing the Arduino IDE and on how to flash the ESP32 with a Sketch.
  3. Clone the Arduino source code from my Github repository: https://github.com/chiwaichan/aws-iot-cat-feeder
  4. Go to the "secrets.h" tab and replace the following variables:

arduino secrets

  • WIFI_SSID: This is the name of your Wifi Access Point
  • WIFI_PASSWORD: The password for your Wifi.
  • AWS_IOT_ENDPOINT: This is the regional endpoint of your AWS Iot Core Service.

aws iot endpoint

  • AWS_CERT_CA: The content of the Amazon Root CA 1 file created in the prerequisites for the first certificate.
  • AWS_CERT_CRT: The content of the xxxxx.cert.pem file created in the prerequisites for the first certificate.
  • AWS_CERT_PRIVATE: The content of the xxxxx.private.key file created in the prerequisites for the first certificate.
  1. Flash the code onto the ESP32

arduino flash code

You might need to push a button on the micro-controller during the flashing process depending on the your ESP32 micro-controller

  1. Check the Arduino console to ensure the ESP32 can connect to AWS IoT and publish messages.

arduino console

  1. Verify the MQTT messages is received by AWS IoT Core

aws iot mqtt test client

Sending a message to the ESP32 when the IoT button is pressed

architecture diagram seeed

The diagram above depicts the components used to send a message to the ESP32 each time the Seeed AWS IoT button is pressed.

  1. AWS IoT button: this is the IoT button I detail in Part 1; it's a physical button that can be anywhere in the world where I can press to feed the fur babies once the final solution is built.
  2. AWS Lambda: This will replace the Lambda from the previous blog with the one shown in the diagram.
  3. IoT Topic: The Lambda will publish a message along with the type of button event (One click, long click or double click) to the Topic "cat-feeder/action", the value of the event is subject to what is supported by the IoT button you use.
  4. AWS IoT Core: This is the service that will forward message to the ESP32 micro-controller that are subscribed to Topics.
  5. ESP32: We will see details of the button event from each click in the Arduino console once this part is set up.

Instructions to set up the AWS IoT button part of the solution

  1. Take the 3 files create in the second set of Certificate created in the AWS IoT Core Service in the prerequisites, then create 3 AWS Secrets Manager "Other type of secret: Plaintext" values. We need a Secret value for each file. This is to provide the Lambda Function the Certificate to call AWS IoT Core.

aws secrets manager

  1. Get a copy of the AWS code from my Github repository: https://github.com/chiwaichan/aws-iot-cat-feeder

  2. In a terminal go into the aws folder and run the commands found in the "sam-commands.text" file, be sure to replace the following values in the commands to reflect the values for your AWS account. This will create a CloudFormation Stack of the AWS IoT Services used by this entire solution.

  • YOUR_BUCKET_NAME
  • Value for IoTEndpoint
  • Value for CatFeederThingLambdaCertName, this is the name of the long certificate value found in Iot Core created in the prerequisites for the second certificate.
  • Value for CatFeederThingLambdaSecretNameCertCA, e.g. "cat-feeder-lambda-cert-ca-aaVaa2", check the name in Secrets Manager.
  • Value for CatFeederThingLambdaSecretNameCertCRT
  • Value for CatFeederThingLambdaSecretNameCertPrivate
  • Value for CatFeederThingControllerCertName, this is the name of the long certificate value found in Iot Core created in the prerequisites for the second certificate used by the ESP32.
  • Find the Lambda created in the CloudFormation stack and Test the Lambda to manually trigger the event.
  • If you have setup an IoT 1-Click Button found in Part 1, you can replace that Lambda with the one created by the CloudFormation Stack. Go to the "AWS IoT 1-Click" Service and edit the "template" for the CatFeeder project.

aws iot one click lambda

  1. Let's press the Iot Button in the following way:
  • Single Click
  • Double Click
  • Long Click
  1. Verify the button events are received by the ESP32 by going to the Arduino console and you should see something like this:

arduino console aws iot mqtt messages

What's next?

I recently got a Creality3D Ender-3 V2 printer, I've got many known unknowns I know I need to get up to speed with in regards to fundamentals of 3D printing and all the tools, techniques and software associated with it. I'll attempt to print an enclosure to house the ESP32 controller, the wires, power supply/battery (if I can source a battery that lasts for more than a month on a single charge) and most importantly the dry cat food; I like to use some mechanical components to dispense food each time we press the IoT button described in Part 1. I'll talk in depth on the progress made on the 3D printing in Part 3.

Smart Cat Feeder – Part 1

· 3 min read
Chiwai Chan
Tinkerer

If you are forgetful when it comes to feeding your fur babies like me, and you often only realise you need to put some dry food into the bowl when you are at work then you should read these series of blogs. Over time, I'll be designing and building a smart cat feeder over time using a combination of components such as Arduino micro controllers, motors, sensors and IoT devices and Cloud services. I'll publish the steps taken in these series of blogs, also, I'll publish any designs and source code as I figure things out and make decisions on aspects of the design.

In this part 1 of the series, I will do a walkthrough on setting up an AWS IoT 1-Click device to trigger a Lambda Function. I got myself one of these Seeed IoT buttons for $20; I also bought a NCR18650B battery which I realised later on is only required if I wanted to run the device without it being powered by a USB type-C cable (used for charging the power as well).

seeed iot button for aws

Firstly, make sure you have an AWS account. Then install the AWS IoT1-Click app onto your phone and log in using your AWS account. With these we will be able to link IoT devices up to our AWS account.

aws iot app login

Claim the IoT device with Device ID

aws iot app claim

Scan the barcode on the back of the device; you can scan multiple devices in bulk.

aws iot app scan aws iot app added aws iot app complete claim aws iot app claim wait for click

Next, I'll set up the Wifi on the device so that it can reach the internet internet from home. Can't see why I can't set it up to my phone's AP for feeding on the go, I'll try it out some other time.

aws iot app wifi

Now we'll create a project and add the IoT device to a placement group in the AWS Console. Give a name and description for the project.

aws iot new project

Next define a template, this is where we create a Lambda function; all the plumbing between the IoT device and Lambda will be handled for us.

aws iot project template

Next we create a placement for the Iot device.

aws iot project placement

aws iot project new placement

aws iot project placement created

Since I have no Arduino micro-controllers (have yet to buy one), I will get the Lambda to log a message.

aws iot lambda

Push the button on the Iot device, wait for the event LED status to turn green after flashing white then check the logs CloudWatch Logs.

aws iot lambda logs

At some point I have to code the Lambda to perform a real action as each event comes through, which will be demonstrated in a following blog in the series instead of just logging to CloudWatch logs.

Within the app on your phone you can see status of each IoT device such as the remaining battery life percentage.

aws iot app devices

As well as a history of the button's events.

aws iot app device events

In the next blog, I'll configure the Lambda to push the event to a Topic for AWS IoT Core to subscribe to, which in turns will trigger an event to an ESP32 ( I've yet to decide on a specific version of the micro-controller) using the IoT MQTT protocol.