You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
treewide: Add documentation for hello_ei sample and Edge Impulse
- Add README for edge_impulse/hello_ei sample.
- Make changes in doc/integrations/edge_impulse.rst related to
change of form of deployment EI models (use of EI SDK module).
- Add section about usage of EI west extension commands.
Jira: NCSDK-37082
Signed-off-by: Bartosz Meus <bartosz.meus@nordicsemi.no>
`Edge Impulse`_ is a development platform that can be used to enable `embedded machine learning`_ on |NCS| devices.
11
-
You can use this platform to collect data from sensors, train machine learning model, and then deploy it to your Nordic Semiconductor's device.
11
+
You can use this platform to collect data from sensors, train a machine learning model, and then deploy it to your Nordic Semiconductor's device.
12
12
13
13
Integration prerequisites
14
14
*************************
@@ -19,16 +19,30 @@ Before you start the |EAI| integration with Edge Impulse, make sure that the fol
19
19
* Setup of the required Development Kit (DK).
20
20
* Creation of an `Edge Impulse studio account <Edge Impulse studio signup_>`_ and an Edge Impulse project.
21
21
22
+
Solution architecture
23
+
*********************
24
+
25
+
The Edge Impulse integration consists of three main components:
26
+
27
+
* Edge Impulse SDK - A C++ library that provides the inference engine and digital signal processing (DSP) functions required to run machine learning models on embedded devices.
28
+
The SDK is integrated into the build system as a Zephyr module through the west manifest file of the |EAI|.
29
+
* Deployed ML model - A Zephyr library package generated by Edge Impulse Studio that contains your trained model's parameters.
30
+
This is provided as a :file:`zip` archive with C and C++ source files that are compiled together with your application.
31
+
* Application code - Your |NCS| application that collects sensor data, feeds it to the inference engine through the Edge Impulse SDK API, and processes the classification results.
32
+
The typical workflow involves collecting sensor samples, running inference using a sliding window approach, and interpreting the classification results to make decisions or trigger actions (see :ref:`hello_ei_sample` sample).
33
+
22
34
Integration overview
23
35
********************
24
36
25
37
Before integrating the Edge Impulse machine learning model into an |EAI| application, you must prepare and deploy the machine learning model for your embedded device.
26
38
This model is prepared using the `Edge Impulse studio`_ external web tool.
27
39
It relies on sensor data that can be provided by different sources, for example data forwarder.
28
-
Check the :ref:`include_ei_data_forwarder_sample` sample for a demonstration of how you can send sensor data to Edge Impulse studio using `Edge Impulse's data forwarder`_.
29
40
30
-
The machine learning model is distributed as a single :file:`zip` archive that includes C++ library sources.
31
-
This file is used by the |NCS| build system to build the Edge Impulse library.
41
+
.. note::
42
+
You can collect data using either a development board that is supported directly by Edge Impulse or your mobile phone.
43
+
Alternatively, you can modify the :ref:`ei_data_forwarder_sample` sample to forward data from a sensor that is connected to any board available in the |NCS|.
44
+
45
+
You must include the deployed machine learning model sources in your |EAI| application.
32
46
33
47
Integration steps
34
48
*****************
@@ -45,46 +59,134 @@ Complete the following steps to generate the archive and add it to the build sys
45
59
Preparing the machine learning model
46
60
====================================
47
61
48
-
To prepare the machine learning model, use `Edge Impulse studio`_ and follow one of the tutorials described in `Edge Impulse getting started guide`_.
49
-
For example, you can try the `Continuous motion recognition tutorial`_.
50
-
This tutorial will guide you through the following steps:
62
+
1. Prepare your own machine learning model.
51
63
52
-
1. Collecting data from sensors and uploading the data to Edge Impulse studio.
64
+
To prepare a machine learning model, use `Edge Impulse studio`_ and follow one of the tutorials described in `Edge Impulse getting started guide`_.
65
+
For example, to test the solution, you can try the `Continuous motion recognition tutorial`_.
66
+
You will complete the following steps:
53
67
54
-
.. note::
55
-
You can use one of the development boards supported directly by Edge Impulse or your mobile phone to collect the data.
56
-
You can also modify the :ref:`include_ei_data_forwarder_sample` application and use it to forward data from a sensor that is connected to any board available in the |NCS|.
68
+
a. Collect data from sensors and upload them to the Edge Impulse studio.
69
+
#. Design your machine learning model (an *impulse*).
70
+
71
+
#. Deploy your machine learning model to use it on an embedded device by following one of the two methods:
72
+
You can obtain a library in two ways:
73
+
74
+
.. tabs::
75
+
76
+
.. group-tab:: Using |EIS| web interface
77
+
78
+
a. Go to the :guilabel:`Deployment` tab and select :guilabel:`Zephyr library`.
79
+
This will generate a :file:`zip` file that contains source files defining the |EI| ML model.
80
+
81
+
.. figure:: ./images/ei_deploy.png
82
+
:scale:50 %
83
+
:alt:Model deployment in |EIS| dashboard
84
+
85
+
Model deployment in |EIS| dashboard
86
+
87
+
.. note::
88
+
Edge Impulse supports multiple deployment formats, some of which are compatible with |NCS| applications.
89
+
However, this instruction focuses on the Zephyr library deployment format.
90
+
91
+
.. group-tab:: Using |EI| west extensions
92
+
93
+
|EI| provides west command extensions that let you build and deploy the machine learning model from |EIS| using the command line instead of the web interface.
94
+
The commands are already configured in the |EAI| west manifest and are ready to use.
95
+
96
+
a. Find two parameters that are required by the commands:
97
+
98
+
* ``Project ID`` - You can find it in the :guilabel:`Project info` panel under :guilabel:`Dashboard`.
99
+
100
+
.. figure:: ./images/ei_project_id.png
101
+
:scale:50 %
102
+
:alt:Project ID in |EIS| dashboard
103
+
104
+
Project ID in |EIS| dashboard
105
+
106
+
* ``API key`` - You can find it under the :guilabel:`Keys` tab in the |EI| project dashboard.
107
+
108
+
.. figure:: ./images/ei_api_key.png
109
+
:scale:50 %
110
+
:alt:API key under the Keys tab in |EIS|
111
+
112
+
API key under the Keys tab in |EIS|
57
113
58
-
#. Designing your machine learning model (an *impulse*).
59
-
#. Deploying the machine learning model to use it on an embedded device.
60
-
As part of this step, you must select the :guilabel:`C++ library` to generate the required :file:`zip` file that contains the source files for building the Edge Impulse library in |EAI|.
114
+
#. Build the machine learning model by running the following command:
115
+
116
+
.. code-block:: console
117
+
118
+
west ei-build -p <PROJECT_ID> -k <API_KEY>
119
+
120
+
#. Deploy the machine learning model by running the following command:
121
+
122
+
.. code-block:: console
123
+
124
+
west ei-deploy -p <PROJECT_ID> -k <API_KEY>
125
+
126
+
As a result, file :file:`ei_model.zip` is downloaded to your working directory.
127
+
128
+
For more details on how to use these commands, see `Automated Deployment with West Commands`_ documentation.
61
129
62
130
.. _ug_edge_impulse_adding_building:
63
131
64
132
.. rst-class:: numbered-step
65
133
66
-
Building the machine learning model in |EAI|
67
-
============================================
134
+
Building an application with machine learning model
After preparing the :file:`zip` archive, you can use the |NCS| build system to build the C++ library with the machine learning model.
70
-
Complete the following steps to configure the building process:
137
+
You have to complete the following configuration steps to be able to build your application including the deployed |EI| machine learning model:
71
138
72
-
1. Make sure that the following Kconfig options are **enabled**:
139
+
1. Make sure that the following Kconfig options are enabled:
73
140
74
141
* ``CONFIG_CPP``
75
142
* ``CONFIG_STD_CPP11``
76
143
* ``CONFIG_REQUIRES_FULL_LIBCPP``
144
+
* ``CONFIG_EDGE_IMPULSE_SDK``
77
145
78
146
.. note::
79
147
The ``CONFIG_FPU`` Kconfig option is implied by default if floating point unit (FPU) is supported by the hardware.
80
148
Using FPU speeds up calculations.
81
149
82
-
#. Make sure that the ``CONFIG_FP16`` Kconfig option is **disabled**.
83
-
The Edge Impulse library is not compatible with half-precision floating point support introduced in Zephyr.
150
+
#. Make sure that the ``CONFIG_FP16`` Kconfig option is disabled.
151
+
The |EI| library is not compatible with half-precision floating point support introduced in Zephyr.
152
+
153
+
#. If you want to call |EI| API directly from C code, you must define the following macros in your application's :file:`CMakeLists.txt` file:
154
+
155
+
.. code-block:: cmake
156
+
157
+
zephyr_compile_definitions(
158
+
EI_C_LINKAGE=1
159
+
EIDSP_SIGNAL_C_FN_POINTER=1
160
+
)
161
+
162
+
Check `Using the library from C`_ for more information.
163
+
164
+
#. Unpack the :file:`zip` archive with the deployed machine learning model and add the following to your application's :file:`CMakeLists.txt` file:
165
+
166
+
.. code-block:: cmake
167
+
168
+
add_subdirectory(ei_model)
169
+
170
+
Alternatively, you can automate the unpacking as part of the build process using CMake's ``FetchContent`` module.
171
+
This approach automatically extracts the archive during configuration:
172
+
173
+
.. code-block:: cmake
174
+
175
+
include(FetchContent)
176
+
177
+
FetchContent_Declare(
178
+
ei_model
179
+
URL ${CMAKE_CURRENT_SOURCE_DIR}/ei_model.zip
180
+
)
181
+
182
+
FetchContent_MakeAvailable(ei_model)
183
+
184
+
.. _ug_edge_impulse_building:
84
185
85
186
Applications and samples
86
187
************************
87
188
88
189
The following samples demonstrate the Edge Impulse integration in the |EAI|:
89
190
90
-
* :ref:`include_ei_data_forwarder_sample` sample - demonstrates how you can send sensor data to Edge Impulse studio using `Edge Impulse's data forwarder`_.
191
+
* :ref:`ei_data_forwarder_sample` sample - Demonstrates how you can send sensor data to |EIS| using `Edge Impulse's data forwarder`_.
192
+
* :ref:`hello_ei_sample` sample - Demonstrates the deployment of models in |EI| and usage of the inference engine provided by |EI| SDK.
.. _`Automated Deployment with West Commands`: https://docs.edgeimpulse.com/hardware/deployments/run-zephyr-module#automated-deployment-with-west-commands-early-access-preview
74
+
.. _`Using the library from C`: https://docs.edgeimpulse.com/hardware/deployments/run-cpp-desktop#using-the-library-from-c
The Hello Edge Impulse sample demonstrates how to use the `Edge Impulse`_ SDK and custom machine learning model when :ref:`integrating Edge Impulse with the nRF Connect SDK <edge_impulse_integration>`.
11
+
12
+
Requirements
13
+
************
14
+
15
+
The sample supports the following development kits:
16
+
17
+
.. table-from-sample-yaml::
18
+
19
+
Overview
20
+
********
21
+
22
+
This sample runs a pre-trained `Edge Impulse`_ machine learning model using two input data series that represent a sine wave and a triangle wave with added noise.
23
+
Sample's main purpose is to:
24
+
25
+
1. Provide input data to the |EI| model.
26
+
#. Start predictions using the machine learning model.
27
+
#. Display the prediction results and time measurements.
28
+
29
+
Configuration
30
+
*************
31
+
32
+
|config|
33
+
34
+
The sample can be configured using the following Kconfig options:
35
+
36
+
.. options-from-kconfig::
37
+
:show-type:
38
+
39
+
Using your custom machine learning model
40
+
========================================
41
+
42
+
If you wish to run this sample using a pre-trained model, go to the :ref:`hello_ei_sample_building_and_running` section.
43
+
Otherwise, if you wish to use a custom machine learning model, you must complete the following setup:
44
+
45
+
1. :ref:`Train and deploy your own machine learning model <ug_edge_impulse_adding_preparing>`.
46
+
Choose |EIS| web interface method (a Zephyr library format).
47
+
48
+
#. Select the |EI| model by setting the ``CONFIG_EDGE_IMPULSE_PATH`` to an absolute or relative path to a file in the local filesystem.
49
+
The file must be a ZIP archive generated by |EIS|, and it must contain the source files that define the |EI| ML model.
50
+
The relative path is tracked from the application source directory (``APPLICATION_SOURCE_DIR``).
51
+
Any CMake variables included in the path are expanded.
52
+
53
+
#. Define input data for the machine learning model in the :file:`samples/edge_impulse/hello_ei/src/include/input_data.h` file.
54
+
#. Check the example input data in your |EIS| project:
55
+
56
+
a. Go to the :guilabel:`Live classification` tab.
57
+
#. In the :guilabel:`Classify existing test sample` panel, select one of the test samples.
58
+
#. Click :guilabel:`Load sample` to display the raw data preview.
59
+
60
+
.. figure:: ./doc/images/ei_load_test_sample.png
61
+
:scale:50 %
62
+
:alt:Loading test sample input data in |EIS|
63
+
64
+
Loading test sample input data in |EIS|
65
+
66
+
The classification results will be displayed along with a preview of raw data.
67
+
68
+
.. figure:: ./doc/images/ei_raw_features.png
69
+
:scale:50 %
70
+
:alt:Raw data preview in |EIS|
71
+
72
+
Raw data preview in |EIS|
73
+
74
+
#. Copy information from the :guilabel:`Raw features` list into an array defined in the :file:`input_data.h` file.
75
+
76
+
.. note::
77
+
If you provide more input data than a single input window can hold, the prediction will be triggered multiple times.
78
+
The input window will be shifted by one input frame between subsequent predictions.
79
+
The prediction will be retriggered until there is no more input data available.
0 commit comments