eval are used on the web, for example, do not typically apply to embedded development. Still, compiling scripts on the device is useful in a handful of situations. We want to provide a clean, maintainable solution for those.
Executive summary: You can now use
eval on embedded devices with XS. Think twice before doing it.
eval on embedded?
eval function is commonly used to compile and execute script source code, and there are several other ways. The XS engine in the Moddable SDK has always supported compilation of source code. However, this capability has only been enabled on desktop builds of XS. This capability of XS has been disabled on embedded devices because:
Compilation requires additional memory. Memory is used when the script is being parsed and compiled to byte code. The byte code itself then resides in RAM until it is no longer needed. For non-trivial scripts, this strains already tight RAM on many embedded devices.
The parsing code increases the size of the XS engine, reducing flash space available for your code.
- Running the parser takes time, slowing the launch of scripts.
eval with XS on embedded
Recent changes to XS apply this same automatic feature stripping to the parser. If the scripts in the firmware use
eval (or another language feature that invoke the parser), then the parser is included. If not, it is automatically removed. This means that no special configuration is required to include the parser in the firmware build -- a script simply invokes it as usual. Importantly, scripts that do not use
eval do not pay the penalty of carrying it in flash.
eval to use Promises either.
In other situations, the developer of the firmware may want to ensure all language features are available. It would be impractical to try to invoke all language features in the application scripts to force the features to be available. The removal of language features is controlled by the project's manifest file. The base manifest for most projects is
manifest_base.json, which uses the following line to declare all unused language features should be removed:
A project that does not want to strip unused features overrides it as follows:
strip feature of the manifest can also be used to explicitly exclude only certain functions. This is more difficult to do, as it requires knowledge of XS internals. An example is given in the Manifest documentation.
Embedded software developers always talk about numbers. This is because the devices that their software executes on often have extremely limited resources, with most relevant here being RAM and Flash (ROM). The RAM and ROM used by XS vary depending on the build configuration, target device, and other factors. To give some perspective, let's take a look at XS flash use on an ESP32 microcontroller built using GCC. Results for other microcontrollers and toolchains should be roughly similar.
Flash code size
To establish a baseline, the hello_world application from the Espressif IDF SDK was built. This generates a binary of 137,008 bytes. This is without XS. We take this as the minimum baseline size of the ESP32 firmware.
A build of helloworld with the
debugger statement invoked by
eval, to force the parser to be included, increases the binary size to 365,136 bytes. Therefore, the parser code size is 57,440 bytes.
Finally, a simple REPL application from the Moddable SDK was built. Its manifest disables automatic removal of unused features giving scripts executed by
eval access to the full language. The resulting binary size is 519,744 bytes, which means that the full XS increased the Flash memory used by 212,048 bytes. Not all of these bytes are used by XS, however. In the hello world example, all functions of the global
Math were stripped. For example,
Math.sin is removed which allows the native linker to remove the C library
sin function too. This means that some of the 212,048 bytes are not strictly part of XS, but the supporting device firmware.
On ESP32, a full deployment of XS, including the parser and
RegExp, is comfortably under 400 KB.
The RAM requirements of the parser vary depending on the script being parsed. As a rule of thumb, having at least 8 KB free RAM is a good idea. The longest individual string or symbol that can be parsed is configured in the manifest. The default is 1 KB, which is enough for most purposes, and may be increased if needed:
xsbug, the XS debugger, has been enhanced to display in the instrumentation panel the number of bytes used by the parser when it executes. This is useful both to see when the parser is invoked and to understand the amount of memory used by the parser during execution.
However, the majority of RAM used by the parser often is in the byte code it outputs, not the temporary memory used during parsing. The byte code resides in the chunk heap. The more scripts compiled, the less RAM is free for other purposes.
Build and run
mcconfig -m -p esp32
When the REPL launches, the ESP32 IDF turns the terminal window into a serial console. You can then type commands to the REPL interactively:
Moddable REPL (version 0.0.1)
> var x = 12
> x + 5
> x ** 2
> eval("x + 3")
The REPL runs on ESP8266:
mcconfig -m -p esp
On ESP8266, you need to run your own terminal program as the default build tools do not provide one. The serial port runs at 921,600 baud, 8 bits, no parity, one stop bit. You may change the serial configuration by modifying the call to
The REPL runs on Mac:
mcconfig -m -p x-cli-mac
The REPL runs on Windows:
mcconfig -m -p x-cli-win
The explicit invocation of
repl on Mac and Windows is necessary as
mcconfig does not launch it directly.
Note: The REPL on Windows does not support arrow key input, so type carefully.
Exploring with the REPL
require function to access modules:
> const Timer = require("timer")
> Timer.set(() => console.log("hello"), 1000)
Note: The Timer module is available in the REPL for embedded devices, but not on Mac or Windows.
cache property of the
require function contains a list of loaded modules, including those preloaded:
To experiment with other modules, add them to the REPL manifest and rebuild. For example, to try
Here's a sample use of the
Base64 module. Note that
encode accepts either a
decode always returns an
> const Base64 = require("base64")
> let result = Base64.decode("eHl6enk=")
> result = new Uint8Array(result)
> result.forEach(value => console.log(String.fromCharCode(value)))
The REPL runs in strict mode. The Moddable SDK always runs in strict mode on embedded devices. Among other things, this means variables must be explicitly declared. The following session shows an example.
ReferenceError: ?: get test: undefined variable
> global.test = 1
> let test = 2
Note: XS implements the
global global variable, a Stage 3 ECMAScript proposal. This global is expected to be renamed to eliminate conflicts with existing libraries on the web. When it is renamed, XS will adopt the new name.
eval on microcontrollers that sell for just a few dollars is unprecedented. It is tempting to use
eval in many places. Yet, we don't recommend its broad use. It is our experience that precompiling on a development machine is a better solution in most circumstances. Dynamically compiling and executing scripts on the device uses considerably more RAM for anything but trivial scripts. It also takes more time and reduces the flash space for your code.
Here are some scenarios where we believe using
eval on the device is a good choice:
Exploration. Being able to interact with the virtual machine interactively is a great way to understand the implementation and the device's capabilities. Using our simple REPL we've already found (and fixed!) a few bugs.
Education. For students getting started with a new language, being able to execute it directly provides immediate feedback.
- Development. During development, especially while debugging, it can be useful to run a few lines of code that aren't part of the executable.
It may be tempting to use the REPL as the start of a command line interface. We don't recommend it. A true command line is generally easier to work with. The Moddable SDK provides a CLI class to add commands to both our serial console and telnet implementations. Not only is this better in most situations where command line interface is used, but the code size and RAM use are considerably lower.