Skip to content

Commit f6a69d2

Browse files
annwojannwoj
committed
doc: add operators page
added operators page Signed-off-by: Anna Wojdylo <anna.wojdylo@nordicsemi.no>
1 parent 7f08580 commit f6a69d2

2 files changed

Lines changed: 306 additions & 0 deletions

File tree

‎doc/index.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,7 @@ See the following documentation:
4444

4545
solution_comparison.rst
4646
quick_start.rst
47+
supported_operators.rst
4748
setting_up_environment.rst
4849
applications.rst
4950
samples.rst

‎doc/supported_operators.rst‎

Lines changed: 305 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,305 @@
1+
.. _supported_operators:
2+
3+
Supported operators
4+
###################
5+
6+
.. contents::
7+
:local:
8+
:depth: 2
9+
10+
This section describes the model structure constraints and the set of operators supported by the Axon Compiler.
11+
12+
Model structure
13+
***************
14+
15+
The Axon Compiler has the following constraints on model structure:
16+
17+
* Supports 8-bit quantized input and output for all layers, with an option to use int32 model output with a configurable radix.
18+
* Supports stateful behavior between inferences when declared using VarHandle, ReadVariable, or AssignVariable.
19+
* Allows a maximum of two inputs per node.
20+
* Supports maximum tensor sizes:
21+
22+
* Height: 1024
23+
* Width: 1024
24+
* Channels: 1024
25+
26+
* Provides native activation functions: ReLU, ReLU6, and LeakyReLU.
27+
* Provides CPU activation functions: Sigmoid, Tanh, and Softmax.
28+
29+
.. _supported_operators_reshape:
30+
31+
Memory organization and reshape
32+
*******************************
33+
34+
Axon stores tensors in memory using the following layout:
35+
36+
.. code-block:: c
37+
38+
int8_t my_axon_tensor[channel_count][height][width];
39+
40+
This differs from TFLite, where channels are the innermost dimension:
41+
42+
.. code-block:: c
43+
44+
int8_t my_tflite_tensor[height][width][channel_count];
45+
46+
In addition, the Axon hardware aligns the start of each output row to a 32‑bit boundary.
47+
When the tensor width is not a multiple of four, this alignment introduces padding bytes at the end of each row.
48+
Because of these differences, a reshape operation in TFLite, which only changes tensor dimensions without moving data, may require actual data movement on Axon.
49+
In such cases, the reshape is executed on the CPU to perform the required memory reorganization.
50+
51+
Many reshape operations are transparent to Axon and do not require data movement, including:
52+
53+
* Dropping or adding an axis when transitioning between 1D and 2D operators.
54+
* Flattening a multi‑channel output before a fully connected operator.
55+
The Axon compiler reorders the weights to account for the channel layout.
56+
* Applying a height/width transpose around an operation (for example, rotating the input by 90 degrees, performing the operation, and then rotating the output back).
57+
In this case, the Axon compiler removes the reshape operations and executes the operation in the unrotated orientation.
58+
59+
.. _supported_operators_list:
60+
61+
Operators
62+
*********
63+
64+
Operators can run entirely on Axon, entirely on the CPU, or be split between the two.
65+
Operators executed on the CPU consume CPU cycles and add a small amount of interrupt latency when they appear in the middle of the model, rather than at the beginning or end.
66+
The following operators are supported in the current version of Axon Compiler:
67+
68+
Convolution operators
69+
=====================
70+
71+
.. list-table::
72+
:header-rows: 1
73+
74+
* - Operator
75+
- Notes / limitations
76+
- Target
77+
- Compiler version
78+
* - Conv1D
79+
- | Dilation not supported.
80+
| Implemented as a 2D convolution with either height or width = 1.
81+
| Max filter width: 32 (when height is 1)
82+
| Max filter height: 16 (when width is 1)
83+
| Max stride: 31
84+
- Axon NPU
85+
- 1.0.0
86+
* - Depthwise Conv1D
87+
- | Max filter width: 32
88+
| Max filter height: 16
89+
| Max stride: 31
90+
- Axon NPU
91+
- 1.0.0
92+
* - Conv2D
93+
- | Dilation supported with width dilation <= 31, and output width 1.
94+
| Max filter dimensions: 16 x 16
95+
| Max stride: 31
96+
- Axon NPU
97+
- 1.0.0
98+
* - Depthwise Conv2D
99+
- | Channel multipliers not supported
100+
| Dilation not supported
101+
| Max filter dimensions: 16 x 16
102+
| Max stride: 31
103+
- Axon NPU
104+
- 1.0.0
105+
106+
Fully connected layer
107+
=====================
108+
109+
.. list-table::
110+
:header-rows: 1
111+
112+
* - Operator
113+
- Notes / limitations
114+
- Target
115+
- Compiler version
116+
* - Fully Connected
117+
- | Maximum input vector length: 2048
118+
| Maximum number of neurons: 2048
119+
- Axon NPU
120+
- 1.0.0
121+
122+
Pooling operators
123+
=================
124+
125+
.. list-table::
126+
:header-rows: 1
127+
128+
* - Operator
129+
- Notes / limitations
130+
- Target
131+
- Compiler version
132+
* - Average Pooling
133+
- | No padding
134+
| Max filter dimensions: 32 x 32
135+
- Axon NPU
136+
- 1.0.0
137+
* - Max Pooling
138+
- | Max filter dimensions: 32 x 32
139+
| Maximum input/output size: 1024
140+
- Axon NPU
141+
- 1.0.0
142+
* - Mean
143+
- Includes global average pooling functionality
144+
- Axon NPU
145+
- 1.0.0
146+
* - Global Average Pooling
147+
- Implemented internally as Mean
148+
- Axon NPU
149+
- 1.0.0
150+
151+
Activation functions
152+
====================
153+
154+
.. list-table::
155+
:header-rows: 1
156+
157+
* - Operator
158+
- Notes / limitations
159+
- Target
160+
- Compiler version
161+
* - ReLU
162+
- Native activation function
163+
- Axon NPU
164+
- 1.0.0
165+
* - ReLU6
166+
- Native activation function
167+
- Axon NPU
168+
- 1.0.0
169+
* - LeakyReLU
170+
- Native activation function
171+
- Axon NPU
172+
- 1.0.0
173+
* - Sigmoid
174+
- Executed on the CPU
175+
- CPU
176+
- 1.0.0
177+
* - Tanh
178+
- Executed on the CPU
179+
- CPU
180+
- 1.0.0
181+
* - Softmax
182+
- Partially accelerated on Axon NPU, with CPU assistance
183+
- Axon NPU | CPU
184+
- 1.0.0
185+
186+
Elementwise operators
187+
=====================
188+
189+
.. list-table::
190+
:header-rows: 1
191+
192+
* - Operator
193+
- Notes / limitations
194+
- Target
195+
- Compiler version
196+
* - Add
197+
- Vector operation with broadcast on height and/or width
198+
- Axon NPU
199+
- 1.0.0
200+
* - Multiply
201+
- Vector operation with broadcast on height and/or width
202+
- Axon NPU
203+
- 1.0.0
204+
205+
Tensor manipulation operators
206+
=============================
207+
208+
.. list-table::
209+
:header-rows: 1
210+
211+
* - Operator
212+
- Notes / limitations
213+
- Target
214+
- Compiler version
215+
* - Strided Slice
216+
- Max stride: 31
217+
- Axon NPU
218+
- 1.0.0
219+
* - Concatenate
220+
- No additional limitations specified
221+
- Axon NPU
222+
- 1.0.0
223+
* - splitV
224+
- No additional limitations specified
225+
- Axon NPU
226+
- 1.0.0
227+
* - Reshape
228+
- | Performed as a CPU operation in those cases where it is not transparent.
229+
| See :ref:`supported_operators_reshape` for details.
230+
- CPU
231+
- 1.0.0
232+
Model design recommendations
233+
****************************
234+
235+
This section provides practical guidance for designing models that compile successfully and run efficiently on the Axon platform.
236+
237+
Preferred architectures
238+
=======================
239+
240+
When designing models for Axon, favor convolution‑based architectures with the following characteristics:
241+
242+
* Using CNN building blocks, such as:
243+
244+
* Conv1D or Conv2D layers
245+
* Depthwise separable convolutions without channel multipliers
246+
* MaxPooling or AveragePooling layers
247+
* Mean for implementing global average pooling
248+
* Fully connected layers that stay within supported size limits
249+
250+
* Using activation functions optimized for NPU execution:
251+
252+
* ReLU
253+
* ReLU6
254+
* LeakyReLU
255+
256+
Architectural constraints
257+
=========================
258+
259+
To stay within compiler and hardware limits, ensure that the model structure adheres to the following constraints:
260+
261+
* Limiting tensor dimensions - Do not exceed the operator limits listed in the :ref:`supported_operators_list` section.
262+
263+
* Limiting fully connected layers:
264+
265+
* Input vector size to 2048 or less
266+
* Number of neurons to 2048 or less
267+
268+
* Limiting graph complexity:
269+
270+
* Using no more than two inputs per node
271+
* Avoiding complex reshape patterns between convolutional and dense layers
272+
273+
* Avoiding unsupported convolution patterns:
274+
275+
* Dilation, unless the output dimension is 1 x 1
276+
* Depthwise convolution with dilation
277+
278+
Quantization guidance
279+
=====================
280+
281+
To achieve optimal performance and predictable accuracy, train models using 8‑bit quantization awareness when targeting deployment on Axon.
282+
283+
Deployment strategy
284+
===================
285+
286+
For time‑series and embedded use cases, the following architectural patterns are recommended:
287+
288+
* Typical Conv1D‑based pipelines:
289+
290+
* Conv1D → ReLU → Pooling → Conv1D → ReLU → Global Average (Mean) → Dense
291+
292+
* Depthwise‑separable pipelines:
293+
294+
* Depthwise Conv → Pointwise Conv → ReLU → Pooling → Dense
295+
296+
* Fully convolutional models:
297+
298+
* Convolutional layers followed by final Mean aggregation
299+
300+
When replacing recurrent or attention‑based models, consider convolutional alternatives such as:
301+
302+
* Temporal convolutional networks without dilation
303+
* Stacked Conv1D blocks combined with pooling and global averaging
304+
305+
These patterns maximize compatibility, performance, and compilation success on the Axon platform.

0 commit comments

Comments
 (0)