Skip to main content
Version: v1.8

AIPU support of ONNX opset15 operators

note

This file is automatically generated from the ONNX operator definitions and Axelera's AIPU operators acceleration support definitions.

OperatorAIPU Acceleration
AddConstrained
AveragePoolConstrained
BatchNormalizationSupported
ClipConstrained
ConcatConstrained
ConvConstrained
ConvTransposeConstrained
FlattenConstrained
GemmConstrained
GlobalAveragePoolSupported
GlobalMaxPoolSupported
HardSigmoidConstrained
HardSwishSupported
LeakyReluSupported
MatMulConstrained
MaxPoolConstrained
MulConstrained
PReluConstrained
PadConstrained
ReluSupported
ReshapeConstrained
ResizeConstrained
SeluConstrained
SigmoidSupported
SliceConstrained
SoftmaxConstrained
SplitConstrained
SqueezeConstrained
SubConstrained
TanhSupported
TransposeConstrained

Add​

Performs element-wise binary addition (with Numpy-style broadcasting support).

This operator supports multidirectional (i.e., Numpy-style) broadcasting.

(Opset 14 change): Extend supported types to include uint8, int8, uint16, and int16.

Version

This version of the operator has been available since version 14 of the default ONNX operator set.

Parameters

  • A: T First operand.
  • B: T Second operand.

Outputs

  • C : T Result, has same element type as two inputs

Type Constraints

  • T: tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(bfloat16) Constrain input and output types to all numeric tensors.

Axelera's notes for developers

Given an operand with shape [N, C, H, W], Addition is supported with other operands with shape [N, C, H, W], [1, C, 1, 1], and scalars. If operands have the same shapes, they must be non-constant.

AIPU Acceleration Constraints

  • allow_config: A.shape == B.shape and not A.is_constant and not B.is_constant
  • allow_config: len(A.shape)==4 and A.shape[1]==1 and B.shape==(0)
  • allow_config: len(B.shape)==4 and B.shape[1]==1 and A.shape==(0)
  • allow_config: len(A.shape)==4 and A.shape[1]!=1 and (B.shape==(1, A.shape[1], 1, 1) or B.shape==())
  • allow_config: len(B.shape)==4 and B.shape[1]!=1 and (A.shape==(1, B.shape[1], 1, 1) or A.shape==())

AveragePool​

AveragePool consumes an input tensor X and applies average pooling across the tensor according to kernel sizes, stride sizes, and pad lengths. average pooling consisting of computing the average on all values of a subset of the input tensor according to the kernel size and downsampling the data into the output tensor Y for further processing. The output spatial shape will be following:

output_spatial_shape[i] = floor((input_spatial_shape[i] + pad_shape[i] - ((kernel_spatial_shape[i] - 1) * dilations[i] + 1)) / strides_spatial_shape[i] + 1)

or

output_spatial_shape[i] = ceil((input_spatial_shape[i] + pad_shape[i] - ((kernel_spatial_shape[i] - 1) * dilations[i] + 1)) / strides_spatial_shape[i] + 1)

if ceil_mode is enabled

* pad_shape[i] is sum of pads along axis i

auto_pad is a DEPRECATED attribute. If you are using them currently, the output spatial shape will be following when ceil_mode is enabled:

VALID: output_spatial_shape[i] = ceil((input_spatial_shape[i] - ((kernel_spatial_shape[i] - 1) * dilations[i] + 1) + 1) / strides_spatial_shape[i])
SAME_UPPER or SAME_LOWER: output_spatial_shape[i] = ceil(input_spatial_shape[i] / strides_spatial_shape[i])

or when ceil_mode is disabled:

VALID: output_spatial_shape[i] = floor((input_spatial_shape[i] - ((kernel_spatial_shape[i] - 1) * dilations[i] + 1) + 1) / strides_spatial_shape[i])
SAME_UPPER or SAME_LOWER: output_spatial_shape[i] = floor(input_spatial_shape[i] / strides_spatial_shape[i])

And pad shape will be following if SAME_UPPER or SAME_LOWER:

pad_shape[i] = (output_spatial_shape[i] - 1) * strides_spatial_shape[i] + ((kernel_spatial_shape[i] - 1) * dilations[i] + 1) - input_spatial_shape[i]

The output of each pooling window is divided by the number of elements (exclude pad when attribute count_include_pad is zero).

Version

This version of the operator has been available since version 11 of the default ONNX operator set.

Parameters

  • X: T Input data tensor from the previous operator; dimensions for image case are (N x C x H x W), where N is the batch size, C is the number of channels, and H and W are the height and the width of the data. For non image case, the dimensions are in the form of (N x C x D1 x D2 ... Dn), where N is the batch size. Optionally, if dimension denotation is in effect, the operation expects the input data tensor to arrive with the dimension denotation of [DATA_BATCH, DATA_CHANNEL, DATA_FEATURE, DATA_FEATURE ...].
  • auto_pad: string (default is NOTSET) auto_pad must be either NOTSET, SAME_UPPER, SAME_LOWER or VALID. Where default value is NOTSET, which means explicit padding is used. SAME_UPPER or SAME_LOWER mean pad the input so that output_shape[i] = ceil(input_shape[i] / strides[i]) for each axis i. The padding is split between the two sides equally or almost equally (depending on whether it is even or odd). In case the padding is an odd number, the extra padding is added at the end for SAME_UPPER and at the beginning for SAME_LOWER.
  • ceil_mode: int (default is 0) Whether to use ceil or floor (default) to compute the output shape.
  • count_include_pad: int (default is 0) Whether include pad pixels when calculating values for the edges. Default is 0, doesn't count include pad.
  • kernel_shape: list of ints (required) The size of the kernel along each axis.
  • pads: list of ints Padding for the beginning and ending along each spatial axis, it can take any value greater than or equal to 0. The value represent the number of pixels added to the beginning and end part of the corresponding axis. pads format should be as follow [x1_begin, x2_begin...x1_end, x2_end,...], where xi_begin the number of pixels added at the beginning of axis i and xi_end, the number of pixels added at the end of axis i. This attribute cannot be used simultaneously with auto_pad attribute. If not present, the padding defaults to 0 along start and end of each spatial axis.
  • strides: list of ints Stride along each spatial axis. If not present, the stride defaults to 1 along each spatial axis.

Outputs

  • Y : T Output data tensor from average or max pooling across the input tensor. Dimensions will vary based on various kernel, stride, and pad sizes. Floor value of the dimension is used

Type Constraints

  • T: tensor(float16), tensor(float), tensor(double) Constrain input and output types to float tensors.

Axelera's notes for developers

Only AveragePool operators with explicit padding (i.e., auto_pad = "NOTSET") are currently supported. Moreover, due to torch runtime constraints, symmetric padding along each dimension must be at most half of the kernel size along the same dimension. Lastly, note that count_include_pad is only supported equal to 1. If padding is specified for this operator, count_include_pad !=1 may lead to wrong results.

AIPU Acceleration Constraints

  • rule: auto_pad=="NOTSET"
  • rule: pads is None or len(pads)==4
  • rule: pads is None or (pads[0]==pads[2] and pads[1]==pads[3] and pads[0]<=0.5*kernel_shape[0] and pads[1]<=0.5*kernel_shape[1]) or (pads[0]!=pads[2]) or (pads[1]!=pads[3])
  • rule: pads is None or (pads!=[0, 0, 0, 0] and count_include_pad==1) or (pads==[0, 0, 0, 0])

BatchNormalization​

Carries out batch normalization as described in the paper https://arxiv.org/abs/1502.03167. Depending on the mode it is being run, There are five required inputs 'X', 'scale', 'B', 'input_mean' and 'input_var'. Note that 'input_mean' and 'input_var' are expected to be the estimated statistics in inference mode (training_mode=False, default), and the running statistics in training mode (training_mode=True). There are multiple cases for the number of outputs, which we list below:

  • Output case #1: Y, running_mean, running_var (training_mode=True)
  • Output case #2: Y (training_mode=False)

When training_mode=False, extra outputs are invalid. The outputs are updated as follows when training_mode=True:

running_mean = input_mean * momentum + current_mean * (1 - momentum)
running_var = input_var * momentum + current_var * (1 - momentum)

Y = (X - current_mean) / sqrt(current_var + epsilon) * scale + B

where:

current_mean = ReduceMean(X, axis=all_except_channel_index)
current_var =  ReduceVar(X, axis=all_except_channel_index)

Notice that ReduceVar refers to the population variance, and it equals to sum(sqrd(x_i - x_avg)) / N where N is the population size (this formula does not use sample size N - 1).

The computation of ReduceMean and ReduceVar uses float to avoid overflow for float16 inputs.

When training_mode=False:

Y = (X - input_mean) / sqrt(input_var + epsilon) * scale + B

For previous (depreciated) non-spatial cases, implementors are suggested to flatten the input shape to (N x C * D1 * D2 * ... * Dn) before a BatchNormalization Op. This operator has optional inputs/outputs. An empty string may be used in the place of an actual argument's name to indicate a missing argument. Trailing optional arguments (those not followed by an argument that is present) may also be simply omitted.

Version

This version of the operator has been available since version 15 of the default ONNX operator set.

Parameters

  • X: T Input data tensor from the previous operator; dimensions are in the form of (N x C x D1 x D2 ... Dn), where N is the batch size, C is the number of channels. Statistics are computed for every channel of C over N and D1 to Dn dimensions. For image data, input dimensions become (N x C x H x W). The op also accepts single dimension input of size N in which case C is assumed to be 1
  • scale: T1 Scale tensor of shape (C).
  • B: T1 Bias tensor of shape (C).
  • input_mean: T2 running (training) or estimated (testing) mean tensor of shape (C).
  • input_var: T2 running (training) or estimated (testing) variance tensor of shape (C).
  • epsilon: float (default is 1e-05) The epsilon value to use to avoid division by zero.
  • momentum: float (default is 0.9) Factor used in computing the running mean and variance.e.g., running_mean = running_mean * momentum + mean * (1 - momentum).
  • training_mode: int (default is 0) If set to true, it indicates BatchNormalization is being used for training, and outputs 1 and 2 are to be computed.

Outputs

  • Y : T The output tensor of the same shape as X
  • running_mean : T2 The running mean after the BatchNormalization operator.
  • running_var : T2 The running variance after the BatchNormalization operator. This op uses the population size (N) for calculating variance, and not the sample size N-1.

Type Constraints

  • T: tensor(float16), tensor(float), tensor(double), tensor(bfloat16) Constrain input and output types to float tensors.
  • T1: tensor(float16), tensor(float), tensor(double), tensor(bfloat16) Constrain scale and bias types to float tensors.
  • T2: tensor(float16), tensor(float), tensor(double), tensor(bfloat16) Constrain mean and variance types to float tensors.

Axelera's notes for developers

Operator is supported in any configurations.

Clip​

Clip operator limits the given input within an interval. The interval is specified by the inputs 'min' and 'max'. They default to numeric_limits::lowest() and numeric_limits::max(), respectively. When 'min' is greater than 'max', the clip operator sets all the 'input' values to the value of 'max'. Thus, this is equivalent to 'Min(max, Max(input, min))'.

Version

This version of the operator has been available since version 13 of the default ONNX operator set.

Parameters

  • input: T Input tensor whose elements to be clipped
  • min: T Minimum value, under which element is replaced by min. It must be a scalar(tensor of empty shape).
  • max: T Maximum value, above which element is replaced by max. It must be a scalar(tensor of empty shape).

Outputs

  • output : T Output tensor with clipped input elements

Type Constraints

  • T: tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(bfloat16) Constrain input and output types to all numeric tensors.

Axelera's notes for developers

Only Clip operators implementing ReLU6 (min=0, max=6) and HardTanh (min=-1, max=1) are currently supported.

AIPU Acceleration Constraints

  • allow_config: min==0 and max==6
  • allow_config: min==-1 and max==1

Concat​

Concatenate a list of tensors into a single tensor. All input tensors must have the same shape, except for the dimension size of the axis to concatenate on.

Version

This version of the operator has been available since version 13 of the default ONNX operator set.

Parameters

  • inputs: T List of tensors for concatenation
  • axis: int (required) Which axis to concat on. A negative value means counting dimensions from the back. Accepted range is [-r, r-1] where r = rank(inputs)..

Outputs

  • concat_result : T Concatenated tensor

Type Constraints

  • T: tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(bfloat16), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128) Constrain output types to any tensor type.

Axelera's notes for developers

Concatenation is supported along the C, H, or W dimension for 4D feature maps.

AIPU Acceleration Constraints

  • allow_config: all([len(x.shape) == 4 for x in inputs]) and axis in [-3, -2, -1, 1, 2, 3]

Conv​

The convolution operator consumes an input tensor and a filter, and computes the output.

Version

This version of the operator has been available since version 11 of the default ONNX operator set.

Parameters

  • X: T Input data tensor from previous layer; has size (N x C x H x W), where N is the batch size, C is the number of channels, and H and W are the height and width. Note that this is for the 2D image. Otherwise the size is (N x C x D1 x D2 ... x Dn). Optionally, if dimension denotation is in effect, the operation expects input data tensor to arrive with the dimension denotation of [DATA_BATCH, DATA_CHANNEL, DATA_FEATURE, DATA_FEATURE ...].
  • W: T The weight tensor that will be used in the convolutions; has size (M x C/group x kH x kW), where C is the number of channels, and kH and kW are the height and width of the kernel, and M is the number of feature maps. For more than 2 dimensions, the kernel shape will be (M x C/group x k1 x k2 x ... x kn), where (k1 x k2 x ... kn) is the dimension of the kernel. Optionally, if dimension denotation is in effect, the operation expects the weight tensor to arrive with the dimension denotation of [FILTER_OUT_CHANNEL, FILTER_IN_CHANNEL, FILTER_SPATIAL, FILTER_SPATIAL ...]. Assuming zero based indices for the shape array, X.shape[1] == (W.shape[1] * group) == C and W.shape[0] mod G == 0. Or in other words FILTER_IN_CHANNEL multiplied by the number of groups should be equal to DATA_CHANNEL and the number of feature maps M should be a multiple of the number of groups G.
  • B: T Optional 1D bias to be added to the convolution, has size of M.
  • auto_pad: string (default is NOTSET) auto_pad must be either NOTSET, SAME_UPPER, SAME_LOWER or VALID. Where default value is NOTSET, which means explicit padding is used. SAME_UPPER or SAME_LOWER mean pad the input so that output_shape[i] = ceil(input_shape[i] / strides[i]) for each axis i. The padding is split between the two sides equally or almost equally (depending on whether it is even or odd). In case the padding is an odd number, the extra padding is added at the end for SAME_UPPER and at the beginning for SAME_LOWER.
  • dilations: list of ints dilation value along each spatial axis of the filter. If not present, the dilation defaults is 1 along each spatial axis.
  • group: int (default is 1) number of groups input channels and output channels are divided into.
  • kernel_shape: list of ints The shape of the convolution kernel. If not present, should be inferred from input W.
  • pads: list of ints Padding for the beginning and ending along each spatial axis, it can take any value greater than or equal to 0. The value represent the number of pixels added to the beginning and end part of the corresponding axis. pads format should be as follow [x1_begin, x2_begin...x1_end, x2_end,...], where xi_begin the number of pixels added at the beginning of axis i and xi_end, the number of pixels added at the end of axis i. This attribute cannot be used simultaneously with auto_pad attribute. If not present, the padding defaults to 0 along start and end of each spatial axis.
  • strides: list of ints Stride along each spatial axis. If not present, the stride defaults is 1 along each spatial axis.

Outputs

  • Y : T Output data tensor that contains the result of the convolution. The output dimensions are functions of the kernel size, stride size, and pad lengths.

Type Constraints

  • T: tensor(float16), tensor(float16), tensor(float), tensor(double) Constrain input and output types to float tensors.

Axelera's notes for developers

Only Conv operators with explicit padding (i.e., auto_pad = "NOTSET") are currently supported. Only symmetric kernels/strides/dilations with kernel_h * kernel_w < 128 are supported for grouped and depthwise convolutions. The kernel dimensions are read from the weight tensor W (shape [M, C/group, kH, kW]) rather than the optional "kernel_shape" attribute, which ONNX exporters frequently omit (leaving it None and inferring it from W).

AIPU Acceleration Constraints

  • rule: auto_pad == "NOTSET"
  • rule: group == 1 or (W.shape[2] == W.shape[3] and (dilations is None or dilations[0] == dilations[1]) and (strides is None or strides[0] == strides[1]))
  • rule: group == 1 or (dilations is not None and ((W.shape[2] - 1) * dilations[0] + 1) * ((W.shape[3] - 1) * dilations[1] + 1) < 128) or (dilations is None and W.shape[2] * W.shape[3] < 128)

ConvTranspose​

The convolution transpose operator consumes an input tensor and a filter, and computes the output.

If the pads parameter is provided the shape of the output is calculated via the following equation:

output_shape[i] = stride[i] * (input_size[i] - 1) + output_padding[i] + ((kernel_shape[i] - 1) * dilations[i] + 1) - pads[start_i] - pads[end_i]

output_shape can also be explicitly specified in which case pads values are auto generated using these equations:

total_padding[i] = stride[i] * (input_size[i] - 1) + output_padding[i] + ((kernel_shape[i] - 1) * dilations[i] + 1) - output_shape[i] If (auto_pads == SAME_UPPER): pads[start_i] = total_padding[i]/2; pads[end_i] = total_padding[i] - (total_padding[i]/2) Else: pads[start_i] = total_padding[i] - (total_padding[i]/2); pads[end_i] = (total_padding[i]/2).

Version

This version of the operator has been available since version 11 of the default ONNX operator set.

Parameters

  • X: T Input data tensor from previous layer; has size (N x C x H x W), where N is the batch size, C is the number of channels, and H and W are the height and width. Note that this is for the 2D image. Otherwise the size is (N x C x D1 x D2 ... x Dn)
  • W: T The weight tensor that will be used in the convolutions; has size (C x M/group x kH x kW), where C is the number of channels, and kH and kW are the height and width of the kernel, and M is the number of feature maps. For more than 2 dimensions, the weight shape will be (C x M/group x k1 x k2 x ... x kn), where (k1 x k2 x ... x kn) is the dimension of the kernel. The number of channels in the output should be equal to W.shape[1] * group (assuming zero based indices of the shape array)
  • B: T Optional 1D bias to be added to the convolution, has size of M.
  • auto_pad: string (default is NOTSET) auto_pad must be either NOTSET, SAME_UPPER, SAME_LOWER or VALID. Where default value is NOTSET, which means explicit padding is used. SAME_UPPER or SAME_LOWER mean pad the input so that output_shape[i] = input_shape[i] * strides[i] for each axis i. The padding is split between the two sides equally or almost equally (depending on whether it is even or odd). In case the padding is an odd number, the extra padding is added at the end for SAME_UPPER and at the beginning for SAME_LOWER.
  • dilations: list of ints dilation value along each spatial axis of the filter. If not present, the dilation defaults to 1 along each spatial axis.
  • group: int (default is 1) number of groups input channels and output channels are divided into.
  • kernel_shape: list of ints The shape of the convolution kernel. If not present, should be inferred from input W.
  • output_padding: list of ints Additional elements added to the side with higher coordinate indices in the output. Each padding value in "output_padding" must be less than the corresponding stride/dilation dimension. By default, this attribute is a zero vector. Note that this attribute doesn't directly affect the computed output values. It only controls the selection of the computed values, so changing this attribute only adds or removes output elements. If "output_shape" is explicitly provided, "output_padding" does not contribute additional size to "output_shape" but participates in the computation of the needed padding amount. This is also called adjs or adjustment in some frameworks.
  • output_shape: list of ints The shape of the output can be explicitly set which will cause pads values to be auto generated. If output_shape is specified pads values are ignored. See doc for details for equations to generate pads. Note that the output_shape attribute value should not include dimensions for batch size and channels, which are automatically inferred.
  • pads: list of ints Padding for the beginning and ending along each spatial axis, it can take any value greater than or equal to 0. The value represent the number of pixels added to the beginning and end part of the corresponding axis. pads format should be as follow [x1_begin, x2_begin...x1_end, x2_end,...], where xi_begin the number of pixels added at the beginning of axis i and xi_end, the number of pixels added at the end of axis i. This attribute cannot be used simultaneously with auto_pad attribute. If not present, the padding defaults to 0 along start and end of each spatial axis.
  • strides: list of ints Stride along each spatial axis. If not present, the stride defaults to 1 along each spatial axis.

Outputs

  • Y : T Output data tensor that contains the result of the convolution. The output dimensions are functions of the kernel size, stride size, pad lengths and group count. The number of channels in the output should be equal to W.shape[1] * group (assuming zero based indices of the shape array)

Type Constraints

  • T: tensor(float16), tensor(float), tensor(double) Constrain input and output types to float tensors.

Axelera's notes for developers

Only ConvTranspose operators with explicit padding (i.e., auto_pad = "NOTSET") are currently supported. Grouped convolutions and "output_shape" attribute are not supported.

AIPU Acceleration Constraints

  • rule: auto_pad == "NOTSET"
  • rule: group == 1
  • rule: pads is None or pads[:len(pads)//2] == pads[len(pads)//2:]
  • rule: output_shape is None

Flatten​

Flattens the input tensor into a 2D matrix. If input tensor has shape (d_0, d_1, ... d_n) then the output will have shape (d_0 X d_1 ... d_(axis-1), d_axis X d_(axis+1) ... X dn).

Version

This version of the operator has been available since version 13 of the default ONNX operator set.

Parameters

  • input: T A tensor of rank >= axis.
  • axis: int (default is 1) Indicate up to which input dimensions (exclusive) should be flattened to the outer dimension of the output. The value for axis must be in the range [-r, r], where r is the rank of the input tensor. Negative value means counting dimensions from the back. When axis = 0, the shape of the output tensor is (1, (d_0 X d_1 ... d_n), where the shape of the input tensor is (d_0, d_1, ... d_n).

Outputs

  • output : T A 2D tensor with the contents of the input tensor, with input dimensions up to axis flattened to the outer dimension of the output and remaining input dimensions flattened into the inner dimension of the output.

Type Constraints

  • T: tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(bfloat16), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128) Constrain input and output to all tensor types.

Axelera's notes for developers

The Flatten operations is only supported in specific scenario, such as before a Gemm layer, or at the end of a model. Note that, even in the cases where it is supported, Flatten should have axes specified as >= 0.

Gemm​

General Matrix multiplication: https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms#Level_3

  • A' = transpose(A) if transA else A
  • B' = transpose(B) if transB else B

Compute Y = alpha * A' * B' + beta * C, where input tensor A has shape (M, K) or (K, M), input tensor B has shape (K, N) or (N, K), input tensor C is broadcastable to shape (M, N), and output tensor Y has shape (M, N). A will be transposed before doing the computation if attribute transA is non-zero, same for B and transB. This operator supports unidirectional broadcasting (tensor C should be unidirectional broadcastable to tensor A * B). This operator has optional inputs/outputs. An empty string may be used in the place of an actual argument's name to indicate a missing argument. Trailing optional arguments (those not followed by an argument that is present) may also be simply omitted.

Version

This version of the operator has been available since version 13 of the default ONNX operator set.

Parameters

  • A: T Input tensor A. The shape of A should be (M, K) if transA is 0, or (K, M) if transA is non-zero.
  • B: T Input tensor B. The shape of B should be (K, N) if transB is 0, or (N, K) if transB is non-zero.
  • C: T Optional input tensor C. If not specified, the computation is done as if C is a scalar 0. The shape of C should be unidirectional broadcastable to (M, N).
  • alpha: float (default is 1.0) Scalar multiplier for the product of input tensors A * B.
  • beta: float (default is 1.0) Scalar multiplier for input tensor C.
  • transA: int (default is 0) Whether A should be transposed
  • transB: int (default is 0) Whether B should be transposed

Outputs

  • Y : T Output tensor of shape (M, N).

Type Constraints

  • T: tensor(float16), tensor(float), tensor(double), tensor(uint32), tensor(uint64), tensor(int32), tensor(int64), tensor(bfloat16) Constrain input and output types to float/int tensors.

Axelera's notes for developers

Gemm with automatic transposition of the first operand (i.e., transA == 1) is not supported.

AIPU Acceleration Constraints

  • rule: transA == 0

GlobalAveragePool​

GlobalAveragePool consumes an input tensor X and applies average pooling across the values in the same channel. This is equivalent to AveragePool with kernel size equal to the spatial dimension of input tensor.

Version

This version of the operator has been available since version 1 of the default ONNX operator set.

Parameters

  • X: T Input data tensor from the previous operator; dimensions for image case are (N x C x H x W), where N is the batch size, C is the number of channels, and H and W are the height and the width of the data. For non image case, the dimensions are in the form of (N x C x D1 x D2 ... Dn), where N is the batch size.

Outputs

  • Y : T Output data tensor from pooling across the input tensor. The output tensor has the same rank as the input. The first two dimensions of output shape are the same as the input (N x C), while the other dimensions are all 1.

Type Constraints

  • T: tensor(float16), tensor(float), tensor(double) Constrain input and output types to float tensors.

Axelera's notes for developers

Operator is supported in any configurations.

GlobalMaxPool​

GlobalMaxPool consumes an input tensor X and applies max pooling across the values in the same channel. This is equivalent to MaxPool with kernel size equal to the spatial dimension of input tensor.

Version

This version of the operator has been available since version 1 of the default ONNX operator set.

Parameters

  • X: T Input data tensor from the previous operator; dimensions for image case are (N x C x H x W), where N is the batch size, C is the number of channels, and H and W are the height and the width of the data. For non image case, the dimensions are in the form of (N x C x D1 x D2 ... Dn), where N is the batch size.

Outputs

  • Y : T Output data tensor from pooling across the input tensor. The output tensor has the same rank as the input. The first two dimensions of output shape are the same as the input (N x C), while the other dimensions are all 1.

Type Constraints

  • T: tensor(float16), tensor(float), tensor(double) Constrain input and output types to float tensors.

Axelera's notes for developers

Operator is supported in any configurations.

HardSigmoid​

HardSigmoid takes one input data (Tensor<T>) and produces one output data (Tensor<T>) where the HardSigmoid function, y = max(0, min(1, alpha * x + beta)), is applied to the tensor elementwise.

Version

This version of the operator has been available since version 6 of the default ONNX operator set.

Parameters

  • X: T Input tensor
  • alpha: float (default is 0.2) Value of alpha.
  • beta: float (default is 0.5) Value of beta.

Outputs

  • Y : T Output tensor

Type Constraints

  • T: tensor(float16), tensor(float), tensor(double) Constrain input and output types to float tensors.

Axelera's notes for developers

Only HardSigmoid operators with pytorch-like parameters (alpha=1/6, beta=0.5) are supported.

AIPU Acceleration Constraints

  • rule: abs(alpha - 0.166667) < 5e-6
  • rule: abs(beta - 0.5) < 5e-6

HardSwish​

HardSwish takes one input data (Tensor<T>) and produces one output data (Tensor<T>) where the HardSwish function, y = x * max(0, min(1, alpha * x + beta)) = x * HardSigmoid<alpha, beta>(x), where alpha = 1/6 and beta = 0.5, is applied to the tensor elementwise.

Version

This version of the operator has been available since version 14 of the default ONNX operator set.

Parameters

  • X: T Input tensor

Outputs

  • Y : T Output tensor

Type Constraints

  • T: tensor(float16), tensor(float), tensor(double) Constrain input and output types to float tensors.

Axelera's notes for developers

Operator is supported in any configurations.

LeakyRelu​

LeakyRelu takes input data (Tensor<T>) and an argument alpha, and produces one output data (Tensor<T>) where the function f(x) = alpha * x for x < 0, f(x) = x for x >= 0, is applied to the data tensor elementwise.

Version

This version of the operator has been available since version 6 of the default ONNX operator set.

Parameters

  • X: T Input tensor
  • alpha: float (default is 0.01) Coefficient of leakage.

Outputs

  • Y : T Output tensor

Type Constraints

  • T: tensor(float16), tensor(float), tensor(double) Constrain input and output types to float tensors.

Axelera's notes for developers

Operator is supported in any configurations.

MatMul​

Matrix product that behaves like numpy.matmul.

Version

This version of the operator has been available since version 13 of the default ONNX operator set.

Parameters

  • A: T N-dimensional matrix A
  • B: T N-dimensional matrix B

Outputs

  • Y : T Matrix multiply results from A * B

Type Constraints

  • T: tensor(float16), tensor(float), tensor(double), tensor(uint32), tensor(uint64), tensor(int32), tensor(int64), tensor(bfloat16) Constrain input and output types to float/int tensors.

Axelera's notes for developers

Matmul is supported only when it is part of the attention block used by the YOLO11 family of networks.

MaxPool​

MaxPool consumes an input tensor X and applies max pooling across the tensor according to kernel sizes, stride sizes, and pad lengths. max pooling consisting of computing the max on all values of a subset of the input tensor according to the kernel size and downsampling the data into the output tensor Y for further processing. The output spatial shape is calculated differently depending on whether explicit padding is used, where pads is employed, or auto padding is used, where auto_pad is utilized. With explicit padding (https://pytorch.org/docs/stable/generated/torch.nn.MaxPool2d.html?highlight=maxpool#torch.nn.MaxPool2d):

output_spatial_shape[i] = floor((input_spatial_shape[i] + pad_shape[i] - dilation[i] * (kernel_shape[i] - 1) - 1) / strides_spatial_shape[i] + 1)

or

output_spatial_shape[i] = ceil((input_spatial_shape[i] + pad_shape[i] - dilation[i] * (kernel_shape[i] - 1) - 1) / strides_spatial_shape[i] + 1)

if ceil_mode is enabled. pad_shape[i] is the sum of pads along axis i.

auto_pad is a DEPRECATED attribute. If you are using them currently, the output spatial shape will be following when ceil_mode is enabled:

VALID: output_spatial_shape[i] = ceil((input_spatial_shape[i] - ((kernel_spatial_shape[i] - 1) * dilations[i] + 1) + 1) / strides_spatial_shape[i])
SAME_UPPER or SAME_LOWER: output_spatial_shape[i] = ceil(input_spatial_shape[i] / strides_spatial_shape[i])

or when ceil_mode is disabled (https://www.tensorflow.org/api_docs/python/tf/keras/layers/AveragePooling2D):

VALID: output_spatial_shape[i] = floor((input_spatial_shape[i] - ((kernel_spatial_shape[i] - 1) * dilations[i] + 1)) / strides_spatial_shape[i]) + 1
SAME_UPPER or SAME_LOWER: output_spatial_shape[i] = floor((input_spatial_shape[i] - 1) / strides_spatial_shape[i]) + 1

And pad shape will be following if SAME_UPPER or SAME_LOWER:

pad_shape[i] = (output_spatial_shape[i] - 1) * strides_spatial_shape[i] + ((kernel_spatial_shape[i] - 1) * dilations[i] + 1) - input_spatial_shape[i]

The output of each pooling window is maximum number of elements exclude pad.

Version

This version of the operator has been available since version 12 of the default ONNX operator set.

Parameters

  • X: T Input data tensor from the previous operator; dimensions for image case are (N x C x H x W), where N is the batch size, C is the number of channels, and H and W are the height and the width of the data. For non image case, the dimensions are in the form of (N x C x D1 x D2 ... Dn), where N is the batch size. Optionally, if dimension denotation is in effect, the operation expects the input data tensor to arrive with the dimension denotation of [DATA_BATCH, DATA_CHANNEL, DATA_FEATURE, DATA_FEATURE ...].
  • auto_pad: string (default is NOTSET) auto_pad must be either NOTSET, SAME_UPPER, SAME_LOWER or VALID. Where default value is NOTSET, which means explicit padding is used. SAME_UPPER or SAME_LOWER mean pad the input so that output_shape[i] = ceil(input_shape[i] / strides[i]) for each axis i. The padding is split between the two sides equally or almost equally (depending on whether it is even or odd). In case the padding is an odd number, the extra padding is added at the end for SAME_UPPER and at the beginning for SAME_LOWER.
  • ceil_mode: int (default is 0) Whether to use ceil or floor (default) to compute the output shape.
  • dilations: list of ints Dilation value along each spatial axis of filter. If not present, the dilation defaults to 1 along each spatial axis.
  • kernel_shape: list of ints (required) The size of the kernel along each axis.
  • pads: list of ints Padding for the beginning and ending along each spatial axis, it can take any value greater than or equal to 0. The value represent the number of pixels added to the beginning and end part of the corresponding axis. pads format should be as follow [x1_begin, x2_begin...x1_end, x2_end,...], where xi_begin the number of pixels added at the beginning of axis i and xi_end, the number of pixels added at the end of axis i. This attribute cannot be used simultaneously with auto_pad attribute. If not present, the padding defaults to 0 along start and end of each spatial axis.
  • storage_order: int (default is 0) The storage order of the tensor. 0 is row major, and 1 is column major. This attribute is used only to convert an n-tuple index value into a single integer value for producing the second output.
  • strides: list of ints Stride along each spatial axis. If not present, the stride defaults to 1 along each spatial axis.

Outputs

  • Y : T Output data tensor from average or max pooling across the input tensor. Dimensions will vary based on various kernel, stride, and pad sizes. Floor value of the dimension is used
  • Indices : I Indices tensor from max pooling across the input tensor. The dimensions of indices are the same as output tensor. The values in indices of are the indices of the selected values during pooling. The indices are computed as flatten 1-D tensor, and the indices do not consider padding. So the values in indices are in [0, N x C x D1 x ... x Dn).

Type Constraints

  • T: tensor(float16), tensor(float), tensor(double), tensor(int8), tensor(uint8) Constrain input and output types to float and 8 bit tensors.
  • I: tensor(int64) Constrain index tensor to int64

Axelera's notes for developers

Only MaxPool operators with explicit padding (i.e., auto_pad = "NOTSET") and row major order (i.e. storage_order=0) are currently supported.

AIPU Acceleration Constraints

  • rule: auto_pad=="NOTSET"
  • rule: storage_order==0

Mul​

Performs element-wise binary multiplication (with Numpy-style broadcasting support).

This operator supports multidirectional (i.e., Numpy-style) broadcasting.

(Opset 14 change): Extend supported types to include uint8, int8, uint16, and int16.

Version

This version of the operator has been available since version 14 of the default ONNX operator set.

Parameters

  • A: T First operand.
  • B: T Second operand.

Outputs

  • C : T Result, has same element type as two inputs

Type Constraints

  • T: tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(bfloat16) Constrain input and output types to all numeric tensors.

Axelera's notes for developers

Given an operand with shape [N, C, H, W], Multiplication is supported with other operands with shape [N, C, H, W], [1, C, 1, 1], and scalars. Multiplication cannot be performed when the left and right operands are the same node.

AIPU Acceleration Constraints

  • allow_config: A.shape == B.shape and not A.is_constant and not B.is_constant
  • allow_config: (A.shape==() and len(B.shape)==4) or (B.shape==() and len(A.shape)==4)
  • allow_config: len(A.shape)==4 and A.shape[1]==1 and B.shape==(0)
  • allow_config: len(B.shape)==4 and B.shape[1]==1 and A.shape==(0)
  • allow_config: len(A.shape)==4 and A.shape[1]!=1 and B.shape==(1, A.shape[1], 1, 1)
  • allow_config: len(B.shape)==4 and B.shape[1]!=1 and A.shape==(1, B.shape[1], 1, 1)

PRelu​

PRelu takes input data (Tensor<T>) and slope tensor as input, and produces one output data (Tensor<T>) where the function f(x) = slope * x for x < 0, f(x) = x for x >= 0., is applied to the data tensor elementwise. This operator supports unidirectional broadcasting (tensor slope should be unidirectional broadcastable to input tensor X).

Version

This version of the operator has been available since version 9 of the default ONNX operator set.

Parameters

  • X: T Input tensor
  • slope: T Slope tensor. The shape of slope can be smaller than first input X; if so, its shape must be unidirectional broadcastable to X

Outputs

  • Y : T Output tensor (same size as X)

Type Constraints

  • T: tensor(float16), tensor(float), tensor(double), tensor(uint32), tensor(uint64), tensor(int32), tensor(int64) Constrain input and output types to float/int tensors.

Axelera's notes for developers

Due to torch runtime constraints, Prelu is supported with either scalar or per-channel slope parameters.

AIPU Acceleration Constraints

  • allow_config: slope.size == 1
  • allow_config: np.array_equal([x for x in slope.shape if x != 1], [X.shape[1]])

Pad​

Given a tensor containing the data to be padded (data), a tensor containing the number of start and end pad values for axis (pads), (optionally) a mode, and (optionally) constant_value, a padded tensor (output) is generated.

The three supported modes are (similar to corresponding modes supported by numpy.pad):

  1. constant(default) - pads with a given constant value as specified by constant_value (which defaults to 0, empty string, or False)

  2. reflect - pads with the reflection of the vector mirrored on the first and last values of the vector along each axis

  3. edge - pads with the edge values of array

Example 1 (constant mode): Insert 0 pads to the beginning of the second dimension.

data = [ [1.0, 1.2], [2.3, 3.4], [4.5, 5.7], ]

pads = [0, 2, 0, 0]

mode = 'constant'

constant_value = 0.0

output = [ [0.0, 0.0, 1.0, 1.2], [0.0, 0.0, 2.3, 3.4], [0.0, 0.0, 4.5, 5.7], ]

Example 2 (reflect mode): data = [ [1.0, 1.2], [2.3, 3.4], [4.5, 5.7], ]

pads = [0, 2, 0, 0]

mode = 'reflect'

output = [ [1.0, 1.2, 1.0, 1.2], [2.3, 3.4, 2.3, 3.4], [4.5, 5.7, 4.5, 5.7], ]

Example 3 (edge mode): data = [ [1.0, 1.2], [2.3, 3.4], [4.5, 5.7], ]

pads = [0, 2, 0, 0]

mode = 'edge'

output = [ [1.0, 1.0, 1.0, 1.2], [2.3, 2.3, 2.3, 3.4], [4.5, 4.5, 4.5, 5.7], ]

Version

This version of the operator has been available since version 13 of the default ONNX operator set.

Parameters

  • data: T Input tensor.
  • pads: tensor(int64) Tensor of integers indicating the number of padding elements to add or remove (if negative) at the beginning and end of each axis. For 2D input tensor, it is the number of pixels. pads should be a 1D tensor of shape [2 * input_rank]. pads format should be: [x1_begin, x2_begin,...,x1_end, x2_end,...], where xi_begin is the number of pad values added at the beginning of axis i and xi_end, the number of pad values added at the end of axis i.
  • constant_value: T (Optional) A scalar value to be used if the mode chosen is constant (by default it is 0, empty string or False).
  • mode: string (default is constant) Supported modes: constant(default), reflect, edge

Outputs

  • output : T Tensor after padding.

Type Constraints

  • T: tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(bfloat16), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128) Constrain input and output types to all tensor types.

Axelera's notes for developers

Pad is currently supported only for the "constant" mode. Moreover, padding along the batch dimension is not supported, and should be specified as 0 in the pads parameter. Padding along the channel dimension should be a multiple of 64.

AIPU Acceleration Constraints

  • rule: constant_value is None or constant_value == 0.0
  • rule: mode not in ["reflect", "edge"]
  • rule: len(pads) == 2 * len(data.shape) and pads[0] == pads[len(data.shape)] == 0
  • rule: len(pads) == 2 * len(data.shape) and pads[1] % 64 == pads[1 + len(data.shape)] % 64 == 0

Relu​

Relu takes one input data (Tensor<T>) and produces one output data (Tensor<T>) where the rectified linear function, y = max(0, x), is applied to the tensor elementwise.

Version

This version of the operator has been available since version 14 of the default ONNX operator set.

Parameters

  • X: T Input tensor

Outputs

  • Y : T Output tensor

Type Constraints

  • T: tensor(float), tensor(int32), tensor(int8), tensor(int16), tensor(int64), tensor(float16), tensor(double), tensor(bfloat16) Constrain input and output types to signed numeric tensors.

Axelera's notes for developers

Operator is supported in any configurations.

Reshape​

Reshape the input tensor similar to numpy.reshape. First input is the data tensor, second input is a shape tensor which specifies the output shape. It outputs the reshaped tensor. At most one dimension of the new shape can be -1. In this case, the value is inferred from the size of the tensor and the remaining dimensions. A dimension could also be 0, in which case the actual dimension value is unchanged (i.e. taken from the input tensor). If 'allowzero' is set, and the new shape includes 0, the dimension will be set explicitly to zero (i.e. not taken from input tensor). Shape (second input) could be an empty shape, which means converting to a scalar. The input tensor's shape and the output tensor's shape are required to have the same number of elements.

If the attribute 'allowzero' is set, it is invalid for the specified shape to contain both a zero value and -1, as the value of the dimension corresponding to -1 cannot be determined uniquely.

Version

This version of the operator has been available since version 14 of the default ONNX operator set.

Parameters

  • data: T An input tensor.
  • shape: tensor(int64) Specified shape for output.
  • allowzero: int (default is 0) (Optional) By default, when any value in the 'shape' input is equal to zero the corresponding dimension value is copied from the input tensor dynamically. allowzero=1 indicates that if any value in the 'shape' input is set to zero, the zero value is honored, similar to NumPy.

Outputs

  • reshaped : T Reshaped data.

Type Constraints

  • T: tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(bfloat16), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128) Constrain input and output types to all tensor types.

Axelera's notes for developers

Reshape is supported in two cases: When used within attention blocks in Yolo11 networks and for trivial operations where the shape parameter equals the input shape (no-op) or acts like a squeeze operation (e.g., [1, N, 1, 1] → [1, N]).

AIPU Acceleration Constraints

  • allow_config: allowzero == 0 and np.array_equal(shape, data.shape)
  • allow_config: allowzero == 0 and len(data.shape) >= 2 and len(shape) == 2 and shape[0] == data.shape[0] and (shape[1] == data.shape[1] or shape[1] == -1)

Resize​

Resize the input tensor. In general, it calculates every value in the output tensor as a weighted average of neighborhood (a.k.a. sampling locations) in the input tensor. Each dimension value of the output tensor is: output_dimension = floor(input_dimension * (roi_end - roi_start) * scale) if input "sizes" is not specified.

Version

This version of the operator has been available since version 13 of the default ONNX operator set.

Parameters

  • X: T1 N-D tensor
  • roi: T2 1-D tensor given as [start1, ..., startN, end1, ..., endN], where N is the rank of X. The RoIs' coordinates are normalized in the coordinate system of the input image. It only takes effect when coordinate_transformation_mode is "tf_crop_and_resize"
  • scales: tensor(float) The scale array along each dimension. It takes value greater than 0. If it's less than 1, it's sampling down, otherwise, it's upsampling. The number of elements of 'scales' should be the same as the rank of input 'X'. One of 'scales' and 'sizes' MUST be specified and it is an error if both are specified. If 'sizes' is needed, the user can use an empty string as the name of 'scales' in this operator's input list.
  • sizes: tensor(int64) The size of the output tensor. The number of elements of 'sizes' should be the same as the rank of input 'X'. Only one of 'scales' and 'sizes' can be specified.
  • coordinate_transformation_mode: string (default is half_pixel) This attribute describes how to transform the coordinate in the resized tensor to the coordinate in the original tensor. The coordinate of each dimension is transformed individually. Let's describe a case using axis x as an example. Denote x_resized as the coordinate of axis x in the resized tensor, x_original as the coordinate of axis x in the original tensor, length_original as the length of the original tensor in axis x, length_resized as the length of the resized tensor in axis x, roi_x = (start_x, end_x) of the axis x in input "roi", scale = length_resized / length_original, if coordinate_transformation_mode is "half_pixel", x_original = (x_resized + 0.5) / scale - 0.5, if coordinate_transformation_mode is "pytorch_half_pixel", x_original = length_resized > 1 ? (x_resized + 0.5) / scale - 0.5 : 0, if coordinate_transformation_mode is "align_corners", x_original = x_resized * (length_original - 1) / (length_resized - 1), if coordinate_transformation_mode is "asymmetric", x_original = x_resized / scale, if coordinate_transformation_mode is "tf_crop_and_resize", x_original = length_resized > 1 ? start_x * (length_original - 1) + x_resized * (end_x - start_x) * (length_original - 1) / (length_resized - 1) : 0.5 * (start_x + end_x) * (length_original - 1).
  • cubic_coeff_a: float (default is -0.75) The coefficient 'a' used in cubic interpolation. Two common choice are -0.5 (in some cases of TensorFlow) and -0.75 (in PyTorch). Check out Equation (4) in https://ieeexplore.ieee.org/document/1163711 for the details. This attribute is valid only if "mode" is "cubic".
  • exclude_outside: int (default is 0) If set to 1, the weight of sampling locations outside the tensor will be set to 0 and the weight will be renormalized so that their sum is 1.0. The default value is 0.
  • extrapolation_value: float (default is 0.0) When coordinate_transformation_mode is "tf_crop_and_resize" and x_original is outside the range [0, length_original - 1], this value is used as the corresponding output value. Default is 0.0f.
  • mode: string (default is nearest) Three interpolation modes: nearest (default), linear and cubic. The "linear" mode includes linear interpolation for 1D tensor and N-linear interpolation for N-D tensor (for example, bilinear interpolation for 2D tensor). The "cubic" mode includes cubic interpolation for 1D tensor and N-cubic interpolation for N-D tensor (for example, bicubic interpolation for 2D tensor).
  • nearest_mode: string (default is round_prefer_floor) Four modes: round_prefer_floor (default, as known as round half down), round_prefer_ceil (as known as round half up), floor, ceil. Only used by nearest interpolation. It indicates how to get "nearest" pixel in input tensor from x_original, so this attribute is valid only if "mode" is "nearest".

Outputs

  • Y : T1 N-D tensor after resizing

Type Constraints

  • T1: tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(bfloat16), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128) Constrain input 'X' and output 'Y' to all tensor types.
  • T2: tensor(float16), tensor(float), tensor(double) Constrain roi type to float or double.

Axelera's notes for developers

Three coordinate transformation modes are supported: "half_pixel", "pytorch_half_pixel", and "asymmetric". For nearest mode, available options are "round_prefer_floor", "round_prefer_ceil", or "floor". Linear resizing is limited to symmetric, integer scaling factors, while nearest resizing supports any integer scaling factors. Note that the "floor" nearest mode can only be used with asymmetric coordinate transformation.

AIPU Acceleration Constraints

  • rule: roi is None
  • rule: mode in ["nearest", "linear"]
  • rule: coordinate_transformation_mode in ["half_pixel", "pytorch_half_pixel", "asymmetric"]
  • rule: nearest_mode in ["round_prefer_floor", "round_prefer_ceil", "floor"]
  • rule: (mode == "linear" and coordinate_transformation_mode in ["half_pixel", "pytorch_half_pixel"]) or (mode == "nearest" and coordinate_transformation_mode in ["half_pixel", "pytorch_half_pixel"] and nearest_mode in ["round_prefer_floor", "round_prefer_ceil"]) or (mode == "nearest" and coordinate_transformation_mode in ["asymmetric"] and nearest_mode in ["floor"])
  • rule: (mode == "linear" and Y.shape[-2] // X.shape[-2] == Y.shape[-1] // X.shape[-1]) or mode == "nearest"
  • rule: (mode == "linear" and Y.shape[-2] % X.shape[-2] == 0) or (mode == "nearest" and (Y.shape[-2] % X.shape[-2] <= 1 or Y.shape[-2] % X.shape[-2] == X.shape[-2] - 1))
  • rule: (mode == "linear" and Y.shape[-1] % X.shape[-1] == 0) or (mode == "nearest" and (Y.shape[-1] % X.shape[-1] <= 1 or Y.shape[-1] % X.shape[-1] == X.shape[-1] - 1))

Selu​

Selu takes one input data (Tensor<T>) and produces one output data (Tensor<T>) where the scaled exponential linear unit function, y = gamma * (alpha * e^x - alpha) for x <= 0, y = gamma * x for x > 0, is applied to the tensor elementwise.

Version

This version of the operator has been available since version 6 of the default ONNX operator set.

Parameters

  • X: T Input tensor
  • alpha: float (default is 1.67326) Coefficient of SELU default to 1.67326319217681884765625 (i.e., float32 approximation of 1.6732632423543772848170429916717).
  • gamma: float (default is 1.0507) Coefficient of SELU default to 1.05070102214813232421875 (i.e., float32 approximation of 1.0507009873554804934193349852946).

Outputs

  • Y : T Output tensor

Type Constraints

  • T: tensor(float16), tensor(float), tensor(double) Constrain input and output types to float tensors.

Axelera's notes for developers

Selu operators are supported if the alpha and gamma parameters are set to the defaults of 1.67326 and 1.0507, respectively.

AIPU Acceleration Constraints

  • rule: abs(alpha - 1.67326) < 5e-6
  • rule: abs(gamma - 1.0507) < 5e-6

Sigmoid​

Sigmoid takes one input data (Tensor<T>) and produces one output data (Tensor<T>) where the sigmoid function, y = 1 / (1 + exp(-x)), is applied to the tensor elementwise.

Version

This version of the operator has been available since version 13 of the default ONNX operator set.

Parameters

  • X: T Input tensor

Outputs

  • Y : T Output tensor

Type Constraints

  • T: tensor(float16), tensor(float), tensor(double), tensor(bfloat16) Constrain input and output types to float tensors.

Axelera's notes for developers

Operator is supported in any configurations.

Slice​

Produces a slice of the input tensor along multiple axes. Similar to numpy: https://numpy.org/doc/stable/user/basics.indexing.html?highlight=slice#slicing-and-striding

Slice uses the starts, ends, axes and steps inputs to select a sub-tensor of its input data tensor.

An effective starts[i], ends[i], and steps[i] must be computed for each i in [0, ... r-1] where r = rank(input) as follows:

If axes are omitted, they are set to [0, ..., r-1]. If steps are omitted, they are set to [1, ..., 1] of length len(starts)

The effective values are initialized as start[i] = 0, ends[i] = dims[i] where dims are the dimensions of input and steps[i] = 1.

All negative elements of axes are made non-negative by adding r to them, where r =rank(input).

All negative values in starts[i] and ends[i] have dims[axes[i]] added to them, where dims are the dimensions of input. Then start[axes[i]] is the adjusted starts[i] is clamped into the range [0, dims[axes[i]]] for positive stepping and [0, dims[axes[i]]-1] for negative stepping.

The clamping for the adjusted ends[i] depends on the sign of steps[i] and must accommodate copying 0 through dims[axes[i]] elements, so for positive stepping ends[axes[i]] is clamped to [0, dims[axes[i]]], while for negative stepping it is clamped to [-1, dims[axes[i]]-1].

Finally, steps[axes[i]] = steps[i].

For slicing to the end of a dimension with unknown size, it is recommended to pass in INT_MAX when slicing forward and 'INT_MIN' when slicing backward.

Example 1:

data = [
    [1, 2, 3, 4],
    [5, 6, 7, 8],
]
axes = [0, 1]
starts = [1, 0]
ends = [2, 3]
steps = [1, 2]
result = [
    [5, 7],
]

Example 2:

data = [
    [1, 2, 3, 4],
    [5, 6, 7, 8],
]
starts = [0, 1]
ends = [-1, 1000]
result = [
    [2, 3, 4],
]

Version

This version of the operator has been available since version 13 of the default ONNX operator set.

Parameters

  • data: T Tensor of data to extract slices from.
  • starts: Tind 1-D tensor of starting indices of corresponding axis in axes
  • ends: Tind 1-D tensor of ending indices (exclusive) of corresponding axis in axes
  • axes: Tind 1-D tensor of axes that starts and ends apply to. Negative value means counting dimensions from the back. Accepted range is [-r, r-1] where r = rank(data). Behavior is undefined if an axis is repeated.
  • steps: Tind 1-D tensor of slice step of corresponding axis in axes. Negative value means slicing backward. 'steps' cannot be 0. Defaults to 1s.

Outputs

  • output : T Sliced data tensor.

Type Constraints

  • T: tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(bfloat16), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128) Constrain input and output types to all tensor types.
  • Tind: tensor(int32), tensor(int64) Constrain indices to integer types

Axelera's notes for developers

Slice is supported along one axis only, which must be specified as input to the operator. Stepped slice is currently supported only on the channel dimension. Slicing along any axis other than the channel axis requires the number of channels to be a multiple of 64

AIPU Acceleration Constraints

  • rule: axes is not None and len(axes) == 1
  • rule: steps is None or (len(steps) == 1 and (steps == 1 or axes == 1))
  • rule: (axes[0] != 1 and data.shape[1] % 64 == 0) or axes[0] == 1

Softmax​

The operator computes the normalized exponential values for the given input:

Softmax(input, axis) = Exp(input) / ReduceSum(Exp(input), axis=axis, keepdims=1)

The "axis" attribute indicates the dimension along which Softmax will be performed. The output tensor has the same shape and contains the Softmax values of the corresponding input.

Version

This version of the operator has been available since version 13 of the default ONNX operator set.

Parameters

  • input: T The input tensor of rank >= axis.
  • axis: int (default is -1) Describes the dimension Softmax will be performed on. Negative value means counting dimensions from the back. Accepted range is [-r, r-1] where r = rank(input).

Outputs

  • output : T The output values with the same shape as the input tensor.

Type Constraints

  • T: tensor(float16), tensor(float), tensor(double), tensor(bfloat16) Constrain input and output types to float tensors.

Axelera's notes for developers

Softmax is supported only when it is part of the attention block used by the YOLO11 family of networks.

Split​

Split a tensor into a list of tensors, along the specified 'axis'. Lengths of the parts can be specified using input 'split'. Otherwise, the tensor is split to equal sized parts.

Version

This version of the operator has been available since version 13 of the default ONNX operator set.

Parameters

  • input: T The tensor to split
  • split: tensor(int64) Optional length of each output. Values should be >= 0.Sum of the values must be equal to the dim value at 'axis' specified.
  • axis: int (default is 0) Which axis to split on. A negative value means counting dimensions from the back. Accepted range is [-rank, rank-1] where r = rank(input).

Outputs

  • outputs : T One or more outputs forming list of tensors after splitting

Type Constraints

  • T: tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(bfloat16), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128) Constrain input and output types to all tensor types.

Axelera's notes for developers

Split is not supported as the first operator in a model. Split is supported for axis different than 0. Negative axis values are not supported.

AIPU Acceleration Constraints

  • rule: axis > 0

Squeeze​

Remove single-dimensional entries from the shape of a tensor. Takes an input axes with a list of axes to squeeze. If axes is not provided, all the single dimensions will be removed from the shape. If an axis is selected with shape entry not equal to one, an error is raised.

Version

This version of the operator has been available since version 13 of the default ONNX operator set.

Parameters

  • data: T Tensors with at least max(dims) dimensions.
  • axes: tensor(int64) List of integers indicating the dimensions to squeeze. Negative value means counting dimensions from the back. Accepted range is [-r, r-1] where r = rank(data).

Outputs

  • squeezed : T Reshaped tensor with same data as input.

Type Constraints

  • T: tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(bfloat16), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128) Constrain input and output types to all tensor types.

Axelera's notes for developers

The Squeeze operations is only supported in specific scenario, such as before a Gemm layer, or at the end of a model.

Sub​

Performs element-wise binary subtraction (with Numpy-style broadcasting support).

This operator supports multidirectional (i.e., Numpy-style) broadcasting.

(Opset 14 change): Extend supported types to include uint8, int8, uint16, and int16.

Version

This version of the operator has been available since version 14 of the default ONNX operator set.

Parameters

  • A: T First operand.
  • B: T Second operand.

Outputs

  • C : T Result, has same element type as two inputs

Type Constraints

  • T: tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(bfloat16) Constrain input and output types to all numeric tensors.

Axelera's notes for developers

Given an operand with shape [N, C, H, W], Subtraction is supported with other operands with shape [1, C, 1, 1] or scalars.

AIPU Acceleration Constraints

  • allow_config: (A.shape==() and len(B.shape)==4) or (B.shape==() and len(A.shape)==4)
  • allow_config: len(A.shape)==4 and A.shape[1]==1 and B.shape==(0)
  • allow_config: len(B.shape)==4 and B.shape[1]==1 and A.shape==(0)
  • allow_config: len(A.shape)==4 and A.shape[1]!=1 and B.shape==(1, A.shape[1], 1, 1)
  • allow_config: len(B.shape)==4 and B.shape[1]!=1 and A.shape==(1, B.shape[1], 1, 1)

Tanh​

Calculates the hyperbolic tangent of the given input tensor element-wise.

Version

This version of the operator has been available since version 13 of the default ONNX operator set.

Parameters

  • input: T Input tensor

Outputs

  • output : T The hyperbolic tangent values of the input tensor computed element-wise

Type Constraints

  • T: tensor(bfloat16), tensor(float16), tensor(float), tensor(double) Constrain input and output types to float tensors.

Axelera's notes for developers

Operator is supported in any configurations.

Transpose​

Transpose the input tensor similar to numpy.transpose. For example, when perm=(1, 0, 2), given an input tensor of shape (1, 2, 3), the output shape will be (2, 1, 3).

Version

This version of the operator has been available since version 13 of the default ONNX operator set.

Parameters

  • data: T An input tensor.
  • perm: list of ints A list of integers. By default, reverse the dimensions, otherwise permute the axes according to the values given.

Outputs

  • transposed : T Transposed output.

Type Constraints

  • T: tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(bfloat16), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128) Constrain input and output types to all tensor types.

Axelera's notes for developers

Transpose operations that are not the no-op, trivial case (i.e. perm=[0, 1, 2, 3]), are not supported. Transpose with perm=[0, 1, 3, 2] is supported only when it is part of the attention block used by the YOLO11 family of networks.

AIPU Acceleration Constraints

  • rule: perm == [0, 1, 2, 3]