Skip to content

README.md

Latest commit

 

History

History
211 lines (153 loc) · 6.56 KB

README.md

File metadata and controls

211 lines (153 loc) · 6.56 KB
Raw

ExecuTorch Arduino Library

Run PyTorch models on Arduino microcontrollers using ExecuTorch.

This directory contains everything needed to package ExecuTorch as an Arduino library. A build script vendors the runtime sources from this repository into a self-contained library that Arduino users install through the Library Manager or by copying into their libraries folder.

How It Works

PyTorch Model ──► torch.export ──► .pte file ──► model.h (C array)
                                                      │
                                          Arduino Sketch (.ino)
                                          #include <ExecuTorchArduino.h>
                                          #include "model.h"
                                                      │
                                          arduino-cli compile ──► Upload ──► Runs on board

The three pieces

  1. The library (arduino_lib/ExecuTorchArduino/) — the ExecuTorch runtime, CMSIS-NN kernels, and portable ops packaged for the Arduino build system. Generated by build_arduino_library.sh; not checked in.

  2. The model (model.h) — a .pte file converted to a C byte array. Each user brings their own model, exported from PyTorch with the Cortex-M backend.

  3. The sketch (.ino) — a standard Arduino program that loads the model, feeds it input, and reads the output. Uses the native ExecuTorch C++ API (Program::load, Method::execute, etc.).

Supported Boards

Board MCU Status
Arduino Uno Q STM32U585 (Cortex-M33) Tested
Arduino Nano 33 BLE nRF52840 (Cortex-M4F) Untested
Arduino Giga R1 WiFi STM32H747 (Cortex-M7) Untested
Arduino Portenta H7 STM32H747 (Cortex-M7) Untested

CMSIS-NN accelerated ops work on any board with an ARM Cortex-M processor with DSP extensions. Portable ops work on any architecture.

Quick Start

1. Build the Arduino library

cd examples/arduino
./build_arduino_library.sh

This copies the required ExecuTorch sources from the repository into arduino_lib/ExecuTorchArduino/, ready for Arduino.

2. Install the library

Copy the generated library into your Arduino libraries folder:

cp -r arduino_lib/ExecuTorchArduino ~/Arduino/libraries/

Or with arduino-cli:

arduino-cli lib install --zip-path arduino_lib/ExecuTorchArduino

3. Export a model

Export a PyTorch model to .pte format with Cortex-M quantization, then convert to a C header:

python export_model.py --model my_model.pt --target cortex-m33 --output model.h

Or use one of the pre-exported models in the examples/ directory.

4. Write a sketch

#include <ExecuTorchArduino.h>
#include "model.h"

using executorch::extension::BufferDataLoader;
using executorch::runtime::Error;
using executorch::runtime::HierarchicalAllocator;
using executorch::runtime::MemoryAllocator;
using executorch::runtime::MemoryManager;
using executorch::runtime::Method;
using executorch::runtime::MethodMeta;
using executorch::runtime::Program;
using executorch::runtime::Result;
using executorch::runtime::Span;

alignas(16) uint8_t method_pool[64 * 1024];
alignas(16) uint8_t temp_pool[8 * 1024];

void setup() {
  Serial.begin(115200);
  delay(2000);

  executorch::runtime::runtime_init();

  auto loader = BufferDataLoader(model_pte, model_pte_size);
  Result<Program> program = Program::load(&loader);
  if (!program.ok()) {
    Serial.println("Failed to load program");
    return;
  }

  // ... load method, set inputs, execute, read outputs
  // See examples/ for complete working sketches.
}

void loop() {
  // Run inference periodically
  delay(2000);
}

The sketch uses the native ExecuTorch C++ API — the same API used on Linux, Android, and bare-metal targets. No wrapper layer, no Arduino-specific abstractions.

5. Compile and upload

arduino-cli compile --fqbn arduino:zephyr:unoq MySketch
arduino-cli upload  --fqbn arduino:zephyr:unoq -p /dev/ttyUSB0 MySketch

What is inside the library

The build_arduino_library.sh script assembles these components from the ExecuTorch repository:

Component Source in repo Purpose
ET Runtime runtime/executor/, runtime/core/, runtime/kernel/, runtime/platform/ Model loading, memory management, op dispatch
Portable Ops kernels/portable/ Software op implementations (any CPU)
Cortex-M Ops backends/cortex_m/ops/ CMSIS-NN accelerated int8 ops
CMSIS-NN fetched by cmake / Zephyr module ARM's optimized DSP kernels
flatcc third-party/flatcc/ .pte file parsing
flatbuffers third-party/flatbuffers/ Schema headers
c10 runtime/core/portable_type/c10/ Core type definitions

The library uses no external dependencies beyond what the Arduino board core provides.

Arduino-specific patches

The build script applies these patches to make ExecuTorch compile under Arduino's build system:

  1. #include <exception> before <variant> — Arduino's custom <new> header omits <exception>, breaking std::bad_variant_access.

  2. cmake_macros.h stub — c10/torch headers expect a cmake-generated file. The stub defines C10_USING_CUSTOM_GENERATED_MACROS.

  3. platform_stubs.c — provides _Exit() and fprintf() for the LLEXT environment on boards that lack them.

Development

Updating the library

After modifying ExecuTorch sources, regenerate the library:

./build_arduino_library.sh        # rebuild
./build_arduino_library.sh --clean  # remove generated output

Testing

arduino-cli compile --fqbn arduino:zephyr:unoq examples/HelloExecuTorch
arduino-cli upload  --fqbn arduino:zephyr:unoq -p /dev/ttyUSB0 examples/HelloExecuTorch
arduino-cli monitor -p /dev/ttyUSB0 --config baudrate=115200

Publishing to Arduino Library Manager

The library is published by adding its repository URL to the Arduino Library Registry. After the initial registration, new git tags are picked up automatically.

Build Validation

Tested on Arduino Uno Q (STM32U585, Cortex-M33 @ 160 MHz):

  • 390+ source files compile with zero errors
  • Flash: 106 KB used (13% of 786 KB)
  • RAM: 91 KB used (69% of 131 KB)
  • Model: DS-CNN keyword spotting, int8 quantized via CMSIS-NN, 52.6 KB