|
| 1 | +.. _solution_comparison: |
| 2 | + |
| 3 | +Overview |
| 4 | +######## |
| 5 | + |
| 6 | +.. contents:: |
| 7 | + :local: |
| 8 | + :depth: 2 |
| 9 | + |
| 10 | +The |EAI| offers several solutions for running machine learning models on Nordic Semiconductor devices. |
| 11 | +Each solution combines a model training toolchain, a runtime or driver layer, and a hardware execution target (CPU or Axon NPU) into an end-to-end workflow. |
| 12 | +The following diagram illustrates how these layers relate to each other: |
| 13 | + |
| 14 | +.. figure:: images/solution_architecture.png |
| 15 | + :alt: Solution architecture overview |
| 16 | + :align: center |
| 17 | + |
| 18 | + High-level architecture of the |EAI|. |
| 19 | + The `Nordic Edge AI Lab`_ cloud service handles online model development, while the on-device stack, consisting of the |EAILib|, the Axon driver, and the SoC hardware, handles embedded inference. |
| 20 | + |
| 21 | +Solutions are grouped into **basic** and **advanced** categories. |
| 22 | +Basic solutions provide higher-level APIs and integrated toolchains that minimize setup effort, while advanced solutions expose lower-level interfaces for fine-grained control over inference execution and resource usage. |
| 23 | + |
| 24 | +If you are unsure which solution fits your use case, the table below provides a quick comparison. |
| 25 | +Detailed descriptions of each solution follow. |
| 26 | + |
| 27 | +.. list-table:: Solution comparison |
| 28 | + :header-rows: 1 |
| 29 | + :widths: 20 20 15 15 30 |
| 30 | + |
| 31 | + * - Solution |
| 32 | + - Training toolchain |
| 33 | + - Execution target |
| 34 | + - Level of control |
| 35 | + - Best for |
| 36 | + * - :ref:`nRF Edge AI Lib API with Axon <solution_edgeai_axon>` |
| 37 | + - `Nordic Edge AI Lab`_ |
| 38 | + - Axon NPU |
| 39 | + - High-level API |
| 40 | + - NPU-accelerated full ML pipeline with minimal integration effort |
| 41 | + * - :ref:`nRF Edge AI Lib API with Neuton <solution_edgeai_neuton>` |
| 42 | + - `Nordic Edge AI Lab`_ |
| 43 | + - CPU |
| 44 | + - High-level API |
| 45 | + - Broad device compatibility with ultra-low memory footprint |
| 46 | + * - :ref:`Axon driver <solution_axon_driver>` |
| 47 | + - :ref:`Axon NPU TFLite compiler <axon_npu_tflite_compiler>` |
| 48 | + - Axon NPU |
| 49 | + - Low-level driver API |
| 50 | + - Custom inference pipelines, direct NPU control, and advanced optimization |
| 51 | + * - :ref:`Edge Impulse <solution_edge_impulse>` |
| 52 | + - `Edge Impulse studio`_ |
| 53 | + - CPU or Axon NPU |
| 54 | + - High-level API |
| 55 | + - End-to-end ML workflow with visual tools and community ecosystem |
| 56 | + |
| 57 | +Basic solutions |
| 58 | +*************** |
| 59 | + |
| 60 | +Use the following basic solutions to get started quickly. |
| 61 | +They rely on integrated toolchains and higher-level APIs that abstract most of the model deployment and runtime details. |
| 62 | + |
| 63 | +.. _solution_edgeai_axon: |
| 64 | + |
| 65 | +nRF Edge AI Lib API with Axon |
| 66 | +============================= |
| 67 | + |
| 68 | +This solution uses models trained with the `Nordic Edge AI Lab`_ and deploys them through the |EAILib| API onto devices equipped with the `Axon NPU`_. |
| 69 | +The Nordic Edge AI Lab allows for model design, training, and optimization in the cloud. |
| 70 | +On the device, the |EAILib| provides a complete ML pipeline that covers the full path from raw sensor data to actionable results: its DSP module performs feature extraction (windowing, spectral transforms, statistical features), and its NN module runs inference on the Axon NPU. |
| 71 | +A lightweight runtime ties these stages together, so applications only interact with a single high-level API. |
| 72 | + |
| 73 | +Use this solution when you want a full, NPU-accelerated ML pipeline with a standardized API and minimal integration effort. |
| 74 | + |
| 75 | +Key characteristics: |
| 76 | + |
| 77 | +* Models are trained and exported from the `Nordic Edge AI Lab`_ web tooling. |
| 78 | +* The |EAILib| delivers the complete on-device pipeline: signal processing, feature extraction, and neural network inference. |
| 79 | +* Inference runs on the Axon NPU for higher throughput and lower power consumption compared to CPU execution. |
| 80 | +* The API abstracts the entire pipeline, keeping applications model-agnostic. |
| 81 | +* Requires a device with `Axon NPU`_ hardware. |
| 82 | + |
| 83 | +See :ref:`quick_start_nrf_edgeai` to get started. |
| 84 | + |
| 85 | +.. _solution_edgeai_neuton: |
| 86 | + |
| 87 | +nRF Edge AI Lib API with Neuton models |
| 88 | +====================================== |
| 89 | + |
| 90 | +This solution uses Neuton models trained with the `Nordic Edge AI Lab`_ and deploys them through the |EAILib| API. |
| 91 | +As with the Axon variant, the |EAILib| provides the complete on-device ML pipeline, including DSP-based feature extraction and neural network inference, but executes entirely on the CPU. |
| 92 | +This makes the solution compatible with a wide range of Nordic Semiconductor devices, including those without an NPU. |
| 93 | + |
| 94 | +The Neuton models are highly optimized and have a minimal memory footprint. Typical resource requirements for Neuton models are 1--5 KB of RAM and 5--10 KB of NVM. Actual RAM and NVM usage depends on the model complexity and the selected signal-processing pipeline. |
| 95 | + |
| 96 | +Use this solution when you need broad device compatibility, an ultra-small footprint, or when your target hardware does not include an NPU. |
| 97 | + |
| 98 | +Key characteristics: |
| 99 | + |
| 100 | +* Models are trained and exported from the `Nordic Edge AI Lab`_ web tooling. |
| 101 | +* The |EAILib| delivers the complete on-device pipeline: signal processing, feature extraction, and neural network inference. |
| 102 | +* Inference runs on the CPU using the Neuton compute engine inside the |EAILib|. |
| 103 | +* Written in portable C with no external dependencies beyond libc. |
| 104 | +* Supports classification, regression, and anomaly detection use cases. |
| 105 | + |
| 106 | +See :ref:`quick_start_nrf_edgeai` to get started. |
| 107 | + |
| 108 | +.. _solution_edge_impulse: |
| 109 | + |
| 110 | +Edge Impulse |
| 111 | +============ |
| 112 | + |
| 113 | +This solution uses the `Edge Impulse`_ platform to provide an end-to-end machine learning workflow, from data collection and model training through deployment on Nordic Semiconductor devices. |
| 114 | +|EIS| offers a visual development interface for designing signal-processing and ML pipelines (called *Impulses*), and generates a portable C++ library that can be compiled together with your |NCS| application. |
| 115 | +Depending on the target device, inference can run on the CPU or be accelerated by the `Axon NPU`_ for higher throughput and lower power consumption. |
| 116 | + |
| 117 | +Use this solution when you prefer a guided, visual ML workflow, when you need built-in data collection tooling, or when you are already working within the Edge Impulse ecosystem. |
| 118 | + |
| 119 | +Key characteristics: |
| 120 | + |
| 121 | +* Models are trained and exported from `Edge Impulse studio`_. |
| 122 | +* Inference runs on the CPU or on the Axon NPU, depending on the target device. |
| 123 | +* Includes tools for sensor data collection, such as the :ref:`ei_data_forwarder_sample`, and supports data upload from mobile devices. |
| 124 | +* Deployed as a Zephyr library package that integrates directly into the |NCS| build system. |
| 125 | +* Extensive documentation and community-contributed datasets are available through the Edge Impulse platform. |
| 126 | + |
| 127 | +See :ref:`quick_start_edge_impulse` for CPU-based deployment, or :ref:`quick_start_axon_edge_impulse` for NPU-accelerated deployment. |
| 128 | + |
| 129 | +Advanced solutions |
| 130 | +****************** |
| 131 | + |
| 132 | +Use the following advanced solutions when you need lower-level control, custom inference pipelines, or direct access to hardware acceleration features. |
| 133 | +These workflows require more manual configuration but offer finer control over performance and resource usage. |
| 134 | + |
| 135 | +.. _solution_axon_driver: |
| 136 | + |
| 137 | +Axon driver |
| 138 | +=========== |
| 139 | + |
| 140 | +This solution gives you direct access to the Axon NPU through the Axon driver API. |
| 141 | +You compile TensorFlow Lite models with the :ref:`Axon NPU TFLite compiler <axon_npu_tflite_compiler>` and implement custom inference pipelines using the driver's synchronous or asynchronous execution modes. |
| 142 | + |
| 143 | +Use this solution when you need maximum control over inference scheduling, memory management, and NPU resource utilization, or when your application requires custom pre- and post-processing that goes beyond what higher-level APIs provide. |
| 144 | + |
| 145 | +Key characteristics: |
| 146 | + |
| 147 | +* Models are compiled from TensorFlow Lite format using the :ref:`Axon NPU TFLite compiler <axon_npu_tflite_compiler>`. |
| 148 | +* The driver supports both synchronous (blocking) and asynchronous (callback-based) inference. |
| 149 | +* Memory is managed through a shared interlayer buffer, sized to the largest model in the system. |
| 150 | +* Provides a host-based software simulator for development and testing without hardware. |
| 151 | +* Requires a device with `Axon NPU`_ hardware for on-target deployment. |
| 152 | + |
| 153 | +See :ref:`quick_start_axon_driver` to get started. |
0 commit comments