Timezone: Europe/London

.

AboutContactMicropythonESP32Links
Moonlight Systems

OTA updating running software without inturruption

During development of my project, I had several ESPs at various locations.
Whenever I made changes to the software, I had to go and retrieve them and re-program them.
This was less than ideal so I needed a way to update them remotely from the internet. They already connected periodically to the web to upload their data, and went into low power deepsleep mode in between. It was not too difficult to add code to the program to check for updates to the program when connected and download them by requesting a PHP script.
I needed to distinguish between units, so chose to use the mac address, and added another PHP script on the server enabling the code to delete the update after it was downloaded, to prevent downloading it again at the next check. The ESP would request the PHP script with a query string containing its mac address. If a file was present matching that mac address it would be read and echoed as the web page, or if not the echo would be FAIL. The ESP would receive this echo as a response to it's request, and use the data to overwrite program.py. When emerging from deepsleep, the new program would be running, and on the next connection to the server would delete the update file.

This worked, but with one or two drawbacks. I was advised it was a security risk having code on the website that could delete files, and sometimes the updates would crash because they were not rigorously tested, and had code errors, or had been corrupted during download.
The delete part was removed and I added a version number to the update files, which was added to the query string, and each updated program would have an incremented version number. This prevented downloading the same file more than once and I could delete them from the server myself at a later date. With regard to crashes, I added code to create a backup before overwriting program.py, and restoring this backup if the update crashed.

I used this method for quite some time without problems, until one day the ESPs would crash every time they updated.
Upon investigation it transpired the program had become so large that it was creating a memory error when writing to file, so I needed a method to reduce the file size. At first I streamlined the code, removed any un-nescessary comments, moved some large functions to a module, and therefore reduced the file size of program.py enough so that the updates were working again.

A workaround at best, the problem would soon return.
It also revealed another drawback of this method, in that I now had a module, and only program.py could be updated.
As the module was part of the program, it seemed likely that would need updating as development progressed.

So, a complete re-write was required, and I ended up with a class library, available on my github as OTA.py

This library enables you to update several programs or modules at the same time, not just program.py and overcomes the file size problem.
It will create backups of any file already present on the ESP before writing it.
Comes with a standalone Python3 utility for creating the update files and some demo programs to show how it works.

Links

IDE editors for programming ESP boards.
Micropython editors

Micropython functionality and libraries.
Micropython.org

Find the uPython firmware for your board.
uPython firmware for development boards.

ESP32 uPython modules and function examples.
Quick reference for the ESP32

Finding the MAC address

Scanning and connecting to multiple SSIDs

Using an SD card reader breakout board

ESP32 Real Time Clock

The Deepsleep function

Storing variables in the RTC

RTC Tuning

OTA Updating running software without interuption

Future articles

Communication between ESPs

Using an Adafruit GPS module

Hassle free web communication with urequests.py.