Moddable SDK Improvements for ESP32 Projects
Today, we released a major update to improve our support for the popular ESP32 microcontroller. This update includes four major new features:
- Faster, more reliable ESP32 build system
- Upgrade to ESP-IDF v3.3.2, a Long Term Support release from Espressif
- Lightweight NimBLE Bluetooth LE stack for ESP32
- Execution of individual build steps — clean, build, deploy, debug — directly
We are continuously updating the build and deployment tools in the Moddable SDK to make them more useable and more useful. To help you stay up-to-date with those, we've added a new document focused on applying updates.
Fast, Native ESP32 Builds on Windows
The Moddable SDK now uses CMake to build the ESP-IDF to improve build performance and reliability. With today's update, all three host platforms — macOS, Windows, and Linux — support native builds using CMake and Espressif's idf.py utility.
The benefits of the new build system are meaningful on every platform. On macOS and Linux, the new build is slightly faster and eliminates a race condition issue in the ESP-IDF build process that required manually running clean builds twice to successfully complete the build.
The changes are most significant on Windows, where ESP32 builds were previously slow and somewhat awkward. On a 2019 Surface Laptop 3 the new build system completes a clean build in approximately one minute, while clean builds with the previous MinGW-based build system took about four and a half minutes. Further, the new build system is a true native cross-compilation on Windows, eliminating the need to manually switch back and forth between a virtualized Linux environment and the Windows command prompt during the build process.
ESP-IDF Long Term Support Release v3.3.2
Today's update to the Moddable SDK brings support for ESP-IDF v3.3.2 as the foundation for ESP32 projects. This is the current Long Term Support release of the ESP-IDF and will be supported by Espressif through February 2022. Prior to today's update, our builds used ESP-IDF version 3.2.2.
ESP-IDF v3.3.2 comes with significant improvements to system security, stability, and performance. Details are available on the ESP-IDF releases page, but highlights of the improvements between v3.2.2 and v3.3.2 include:
- Dozens of security, stability, and performance fixes
- Fixes for two critical Wi-Fi reliability issues
- Major improvements to the Wi-Fi stack's SoftAP mode
- More refined OTA support
- A new CMake-based build system
- NimBLE Bluetooth stack
By incorporating ESP-IDF v3.3.2 into builds with the Moddable SDK, we are bringing all of these improvements to our developers.
More Efficient Bluetooth LE with NimBLE
Today's update to the Moddable SDK adds support for the Apache NimBLE Bluetooth LE 5.1-certified open-source host on ESP32. The NimBLE stack is new to ESP-IDF v3.3.x and is recommended by Espressif for projects using only BLE (i.e. not using classic Bluetooth). Compared to the dual-mode Bluedroid stack, also included in the ESP-IDF and previously used in the Moddable SDK, NimBLE provides several benefits:
- Smaller Flash footprint. The NimBLE stack has a smaller binary code size than Bluedroid. In a typical BLE project, for example the heart-rate-server, the firmware is about 254 KB smaller. Not only does this free-up limited firmware space for your projects, but reduces the time needed to deploy builds when developing.
- Smaller RAM footprint. The NimBLE stack also uses less memory than Bluedroid. The same BLE heart-rate-server example, after establishing a connection, has 33 KB more system memory free when using NimBLE. As the ESP32 has only about 250 KB of RAM available to your project, this is a significant improvement.
- Support for up to nine simultaneous connections, compared to seven with Bluedroid.
- Fewer buffer copies. NimBLE provides a unified buffering scheme that eliminates buffer copies from the BLE stack to the Moddable SDK.
- Faster builds. The NimBLE code builds approximately 907 ESP-IDF source files compared to 1061 ESP-IDF sources required for Bluedroid.
Execution of Individual Build Steps
Today's update includes an improvement to mcconfig, our command line tool that generates make files based on a manifest and then builds and launches Moddable SDK projects on microcontrollers or in the simulator.
By default, mcconfig performs a complete build, deploy, and debug sequence for the selected platform. For instance, the command
mcconfig -d -m -p esp32 will build your project, flash it to the ESP32 device, and start
serial2xsbug to connect the device to the xsbug debugger on your host machine.
Today's update adds a new optional flag for mcconfig:
-t <target>. By selecting a target, you configure mcconfig to perform only a single step.
The options for the
<target> argument are:
clean: removes build outputs for the project
build: builds the project, but does not deploy it
deploy: deploys a previously built project without rebuilding
xsbug: connect the xsbug debugger to a previously-deployed project
all: performs all steps (This is the default value when the
-t flag is omitted)
These changes to the ESP32 build were so great to have that we applied them across our other target platforms as well. The
build options are applicable across all target platforms (i.e. ESP8266, ESP32, Gecko, QCA4020, and the desktop simulators). The
xsbug option is available for ESP8266, ESP32, and QCA4020. The
deploy option is available for the ESP8266 and ESP32.
This update allows for more flexible development workflows and interoperability with other tools. For instance, a continuous integration tool could use the
-t build option to rebuild a project after each change without attempting to deploy it. Developers can use the
-t deploy option to switch between deploying multiple projects to a device without rebuilding. And it is often useful to restart the debugging process without redeploying by using the
-t xsbug option.
Keeping Up To Date with Moddable SDK Improvements
It is important to maintain an up-to-date development environment to get security and reliability fixes and new features, including those described in this article. The Moddable SDK tools and the utilities used to build for target platforms need to be occasionally updated or rebuilt.
To help you with that process, we have added a new Keeping Up To Date guide to our documentation. Follow along with that document to update your Moddable SDK repository, rebuild the Moddable SDK tools, and install and update the utilities you will need in your build environment for the updated ESP32 build system.
Support for the older ESP32 build systems remains in the Moddable SDK for now (set the environment variable
ESP32_CMAKE=0). But we hope that everyone will move up to the new build system soon to enjoy its many benefits.
Today's Moddable SDK update brings a pile of huge improvements to our developer experience for the wildly-popular ESP32 microcontroller.
ESP32 Moddable SDK developers now have access to:
- Much faster builds on Windows and a consistent ESP32 build experience across host platforms
- Espressif's Long Term Support ESP-IDF v3.3.2, which arrived with a number of critical bug fixes
- Dramatically improved NimBLE Bluetooth LE stack
- Individually-selectable build targets for the
mcconfig build tool
We recommend that all of our developers upgrade their Moddable SDK today using our newly published Keeping Up To Date document. If you have any issues—or if you just love today's update—please reach out to us on GitHub or Twitter.