onnxruntime/docs/ContribOperators.md at 43521c0de70396fc3e56bab82ddd047868e5a60d

mirror of https://github.com/saymrwulf/onnxruntime.git synced 2026-05-14 20:48:00 +00:00

add documentation for custom ops (#708 )

* added tools for doc gen, added doc

* doc updated

* some fixes

* hooked up with build.py

* hooked up with build.py and fail on nonupdated doc

* update

2019-03-26 21:58:01 -07:00

Contrib Operator Schemas

This file is automatically generated from the def files via this script. Do not modify directly and instead edit operator definitions.

com.microsoft

com.microsoft

com.microsoft.AttnLSTM

Computes an one-layer RNN where its RNN Cell is an AttentionWrapper wrapped a LSTM Cell. The RNN layer contains following basic component: LSTM Cell, Bahdanau Attention Mechanism, AttentionWrapp.

Activation functions:

Relu(x)                - max(0, x)

Tanh(x)                - (1 - e^{-2x})/(1 + e^{-2x})

Sigmoid(x)             - 1/(1 + e^{-x})

(NOTE: Below are optional)

Affine(x)              - alpha*x + beta

LeakyRelu(x)           - x if x >= 0 else alpha * x

ThresholdedRelu(x)     - x if x >= alpha else 0

ScaledTanh(x)          - alpha*Tanh(beta*x)

HardSigmoid(x)         - min(max(alpha*x + beta, 0), 1)

Elu(x)                 - x if x >= 0 else alpha*(e^x - 1)

Softsign(x)            - x/(1 + |x|)

Softplus(x)            - log(1 + e^x)

Softmax(x)             - exp(x) / sum(exp(x))

Bahdanau Attention Mechanism: M - Memory tensor.

  `VALUES` - masked Memory by its real sequence length.

  `MW` - Memory layer weight.

  `KEYS` - Processed memory tensor by the memory layer.
           KEYS = M * MW

  `Query` - Query tensor, normally at specific time step in sequence.

  `QW` - Query layer weight in the attention mechanism

  `PQ` - processed query,  = `Query` * `QW`

  `V' - attention vector

  `ALIGN` - calculated alignment based on Query and KEYS
      ALIGN = softmax(reduce_sum(`V` * Tanh(`KEYS` + `PQ`)))

  `CONTEXT` - context based on `ALIGN` and `VALUES`
      CONTEXT = `ALIGN` * `VALUES`

LSTM Cell: X - input tensor concat with attention state in the attention wrapper

`i` - input gate

`o` - output gate

`f` - forget gate

`c` - cell gate

`t` - time step (t-1 means previous time step)

`W[iofc]` - W parameter weight matrix for input, output, forget, and cell gates

`R[iofc]` - R recurrence weight matrix for input, output, forget, and cell gates

`Wb[iofc]` - W bias vectors for input, output, forget, and cell gates

`Rb[iofc]` - R bias vectors for input, output, forget, and cell gates

`P[iof]`  - P peephole weight vector for input, output, and forget gates

`WB[iofc]` - W parameter weight matrix for backward input, output, forget, and cell gates

`RB[iofc]` - R recurrence weight matrix for backward input, output, forget, and cell gates

`WBb[iofc]` - W bias vectors for backward input, output, forget, and cell gates

`RBb[iofc]` - R bias vectors for backward input, output, forget, and cell gates

`PB[iof]`  - P peephole weight vector for backward input, output, and forget gates

`H` - Hidden state

`num_directions` - 2 if direction == bidirectional else 1

Equations (Default: f=Sigmoid, g=Tanh, h=Tanh):

  - it = f(Xt*(Wi^T) + Ht-1*(Ri^T) + Pi (.) Ct-1 + Wbi + Rbi)

  - ft = f(Xt*(Wf^T) + Ht-1*(Rf^T) + Pf (.) Ct-1 + Wbf + Rbf)

  - ct = g(Xt*(Wc^T) + Ht-1*(Rc^T) + Wbc + Rbc)

  - Ct = ft (.) Ct-1 + it (.) ct

  - ot = f(Xt*(Wo^T) + Ht-1*(Ro^T) + Po (.) Ct + Wbo + Rbo)

  - Ht = ot (.) h(Ct)

AttentionWrapp Notations: `lstm()' - wrapped inner cell. Ht, Ct = lstm(concat(Xt, ATTNt-1), Ct-1)

`am()` - attention mechanism the wrapper used.
         CONTEXTt, ALIGNt = am(Ht, ALIGNt-1)

`AW` - attention layer weights, optional.

`ATTN` - attention state, initial is zero. If `AW` provided, it is the output of the attention layer,
              ATTNt = concat(Ht, CONTEXTt) * AW
         otherwise,
              ATTNt = CONTEXTt

RNN layer output: Y - if needed is the sequence of Ht from lstm cell.

`Y_h` - is the last valid H from lstm cell.

`Y_c` - is the last valid C from lstm cell.

Version

This version of the operator has been available since version 1 of the 'com.microsoft' operator set.

Attributes

activation_alpha : list of floats: Optional scaling values used by some activation functions. The values are consumed in the order of activation functions, for example (f, g, h) in LSTM. Default values are the same as of corresponding ONNX operators.For example with LeakyRelu, the default alpha is 0.01.
activation_beta : list of floats: Optional scaling values used by some activation functions. The values are consumed in the order of activation functions, for example (f, g, h) in LSTM. Default values are the same as of corresponding ONNX operators.
activations : list of strings: A list of 3 (or 6 if bidirectional) activation functions for input, output, forget, cell, and hidden. The activation functions must be one of the activation functions specified above. Optional: See the equations for default if not specified.
clip : float: Cell clip threshold. Clipping bounds the elements of a tensor in the range of [-threshold, +threshold] and is applied to the input of activations. No clip if not specified.
direction : string: Specify if the RNN is forward, reverse, or bidirectional. Must be one of forward (default), reverse, or bidirectional.
hidden_size : int: Number of neurons in the hidden layer.
input_forget : int: Couple the input and forget gates if 1, default 0.

Inputs (3 - 14)

:
:
:
(optional) :
(optional) :
(optional) :
(optional) :
(optional) :
(optional) :
(optional) :
(optional) :
(optional) :
(optional) :
(optional) :

Outputs (0 - 3)

(optional) :
(optional) :
(optional) :

Type Constraints

:
:

The integer convolution operator consumes an input tensor, a filter, and a padding value, and computes the output. The production MUST never overflow. The accumulation may overflow if and only if in 32 bits.

Version

This version of the operator has been available since version 1 of the 'com.microsoft' operator set.

Attributes

auto_pad : string: auto_pad must be either NOTSET, SAME_UPPER, SAME_LOWER or VALID. Where default value is NOTSET, which means explicit padding is used. SAME_UPPER or SAME_LOWER mean pad the input so that the output size match the input.In case of odd number add the extra padding at the end for SAME_UPPER and at the beginning for SAME_LOWER. VALID mean no padding.
dilations : list of ints: dilation value along each axis of the filter. If not present, the dilation defaults to 1 along each axis.
group : int: number of groups input channels and output channels are divided into. default is 1.
kernel_shape : list of ints: The shape of the convolution kernel. If not present, should be inferred from input 'w'.
pads : list of ints: Padding for the beginning and ending along each axis, it can take any value greater than or equal to 0.The value represent the number of pixels added to the beginning and end part of the corresponding axis.`pads` format should be as follow [x1_begin, x2_begin...x1_end, x2_end,...], where xi_begin the number ofpixels added at the beginning of axis `i` and xi_end, the number of pixels added at the end of axis `i`.This attribute cannot be used simultaneously with auto_pad attribute. If not present, the padding defaultsto 0 along start and end of each axis.
strides : list of ints: Stride along each axis. If not present, the stride defaults to 1 along each axis.

Inputs (2 - 4)

:
:
(optional) :
(optional) :

Outputs

:

Type Constraints

:
:
:

com.microsoft.DequantizeLinear

The linear de-quantization operator. It consumes a quantized data, a scale, a zero point and computes the full precision data. The dequantization formula is y = (x - x_zero_point) * x_scale. Scale and zero point must have same shape. They must be either scalar (per tensor) or 1-D tensor (per 'axis').

Version

This version of the operator has been available since version 1 of the 'com.microsoft' operator set.

Attributes

axis : int: the axis along which same quantization parameters are applied. It's optional. If it's not specified, it means per-tensor quantization and input 'x_scale' and 'x_zero_point' must be scalars. If it's specified, it means per 'axis' quantization and input 'x_scale' and 'x_zero_point' must be 1-D tensors.

Inputs

:
:
:

Outputs

:

Type Constraints

:
:

com.microsoft.ExpandDims

ExpandDims echo operator.

Version

This version of the operator has been available since version 1 of the 'com.microsoft' operator set.

Inputs

:
:

Outputs

:

Type Constraints

:

com.microsoft.FusedConv

The fused convolution operator schema is the same as Conv besides it includes an attribute activation.

Version

This version of the operator has been available since version 1 of the 'com.microsoft' operator set.

Attributes

activation : string
alpha : float
auto_pad : string
dilations : list of ints
group : int
kernel_shape : list of ints
pads : list of ints
strides : list of ints

Inputs (2 - 3)

:
:
(optional) :

Outputs

:

Type Constraints

:

com.microsoft.FusedGemm

The FusedGemm operator schema is the same as Gemm besides it includes attributes activation and leaky_relu_alpha.

Version

This version of the operator has been available since version 1 of the 'com.microsoft' operator set.

Attributes

activation : string
alpha : float: Scalar multiplier for the product of input tensors A * B.
beta : float: Scalar multiplier for input tensor C.
leaky_relu_alpha : float
transA : int: Whether A should be transposed
transB : int: Whether B should be transposed

Inputs

:
:
:

Outputs

:

Type Constraints

:

com.microsoft.GatherND

Given data tensor of rank r >= 1, and indices tensor of rank q >= 1, gather slices of data into an output tensor of rank q - 1 + r - indices[-1]. Example 1: data = 0,1],[2,3 indices = 0,0],[1,1 output = [0,3] Example 2: data = 0,1],[2,3 indices = 1],[0 output = 2,3],[0,1 Example 3: data = [0,1],[2,3,4,5],[6,7] indices = 0,1],[1,0 output = 2,3],[4,5 Example 4: data = [0,1],[2,3,4,5],[6,7] indices = [0,1,1,0] output = [2,3,4,5]

Version

This version of the operator has been available since version 1 of the 'com.microsoft' operator set.

Inputs

:
:

Outputs

:

Type Constraints

:
:

com.microsoft.MatMulInteger

Matrix product that behaves like numpy.matmul: https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.matmul.html. The production MUST never overflow. The accumulation may overflow if and only if in 32 bits.

Version

This version of the operator has been available since version 1 of the 'com.microsoft' operator set.

Inputs (2 - 4)

:
:
(optional) :
(optional) :

Outputs

:

Type Constraints

:
:
:

com.microsoft.MaxpoolWithMask

For internal use.

Version

This version of the operator has been available since version 1 of the 'com.microsoft' operator set.

Attributes

auto_pad : string
kernel_shape : list of ints
pads : list of ints
storage_order : int
strides : list of ints

Inputs

:
:

Outputs

:

Type Constraints

:

com.microsoft.MurmurHash3

The underlying implementation is MurmurHash3_x86_32 generating low latency 32bits hash suitable for implementing lookup tables, Bloom filters, count min sketch or feature hashing.

Version

This version of the operator has been available since version 1 of the 'com.microsoft' operator set.

Attributes

positive : int: If value is 1, output type is uint32_t, else int32_t. Default value is 1.
seed : int: Seed for the hashing algorithm, unsigned 32-bit integer, default to 0.

Inputs

:

Outputs

:

Type Constraints

:
:

com.microsoft.NonMaxSuppression

Pruning away boxes that have high intersection-over-union (IOU) overlap with previously selected boxes. Bounding boxes with score less than score_threshold are removed. Bounding boxes are supplied as [y1, x1, y2, x2], where (y1, x1) and (y2, x2) are the coordinates of any diagonal pair of box corners and the coordinates can be provided as normalized (i.e., lying in the interval [0, 1]) or absolute. Note that this algorithm is agnostic to where the origin is in the coordinate system and more generally is invariant to orthogonal transformations and translations of the coordinate system; thus translating or reflections of the coordinate system result in the same boxes being selected by the algorithm. The output of this operation is a set of integers indexing into the input collection of bounding boxes representing the selected boxes. The bounding box coordinates corresponding to the selected indices can then be obtained using the gather operation.

Version

This version of the operator has been available since version 1 of the 'com.microsoft' operator set.

Attributes

iou_threshold : float: Float representing the threshold for deciding whether boxes overlap too much with respect to IOU. Value range [0, 1]. The default is 0.0
max_output_size : int (required): Integer representing the maximum number of boxes to be selected by non max suppression.
pad_to_max_output_size : int: Optional. 1(true) - the output selected_indices is padded to be of length max_output_size. Defaults to 0(false).
score_threshold : float (required): Float tensor representing the threshold for deciding when to remove boxes based on score.

Inputs

:
:

Outputs (1 - 2)

:
(optional) :

Type Constraints

:
:

com.microsoft.QLinearConv

The convolution operator consumes a quantized input tensor, its scale and zero point, a quantized filter, its scale and zero point, and output's scale and zero point, and computes the quantized output. Each scale and zero point pair must have same shape. It means they must be either scalars (per tensor) or 1-D tensors (per channel). The production MUST never overflow. The accumulation may overflow in 32 bits if the input is 8 bits or in 64 bits if the input is 16 bits.

Version

This version of the operator has been available since version 1 of the 'com.microsoft' operator set.

Attributes

auto_pad : string: auto_pad must be either NOTSET, SAME_UPPER, SAME_LOWER or VALID. Where default value is NOTSET, which means explicit padding is used. SAME_UPPER or SAME_LOWER mean pad the input so that the output size match the input.In case of odd number add the extra padding at the end for SAME_UPPER and at the beginning for SAME_LOWER. VALID mean no padding.
dilations : list of ints: dilation value along each axis of the filter. If not present, the dilation defaults to 1 along each axis.
group : int: number of groups input channels and output channels are divided into. default is 1.
kernel_shape : list of ints: The shape of the convolution kernel. If not present, should be inferred from input 'w'.
pads : list of ints: Padding for the beginning and ending along each axis, it can take any value greater than or equal to 0.The value represent the number of pixels added to the beginning and end part of the corresponding axis.`pads` format should be as follow [x1_begin, x2_begin...x1_end, x2_end,...], where xi_begin the number ofpixels added at the beginning of axis `i` and xi_end, the number of pixels added at the end of axis `i`.This attribute cannot be used simultaneously with auto_pad attribute. If not present, the padding defaultsto 0 along start and end of each axis.
strides : list of ints: Stride along each axis. If not present, the stride defaults to 1 along each axis.

Inputs (8 - 9)

:
:
:
:
:
:
:
:
(optional) :

Outputs

:

Type Constraints

:
:
:
:

com.microsoft.QLinearMatMul

Matrix product that behaves like numpy.matmul: https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.matmul.html. It consumes two quantized input tensors, their scales and zero points, and output's scale and zero point, and computes the quantized output. The quantization formula is x_quantized = (x_fp32 / x_scale) + x_zero_point. For (x_fp32 / x_scale), it computes the nearest integer value to arg (in floating-point format), rounding halfway cases away from zero. Scale and zero point must have same shape. They must be either scalar (per tensor) or 1-D tensor (per row for a and per column for b). If scale and zero point are 1D tensor, the number of elements of scale and zero point tensor of input 'a' and output 'y' should be equal to the number of rows of input 'a', and the number of elements of scale and zero point tensor of input 'b' should be equal to the number of columns of input 'b'. The production MUST never overflow. The accumulation may overflow in 32 bits if the input is 8 bits or in 64 bits if the input is 16 bits.

Version

This version of the operator has been available since version 1 of the 'com.microsoft' operator set.

Inputs

:
:
:
:
:
:
:
:

Outputs

:

Type Constraints

:
:
:

com.microsoft.QuantizeLinear

The linear quantization operator. It consumes a full precision data, a scale, a zero point and computes the quantized data. The quantization formula is y = (x / y_scale) + y_zero_point. For (x / y_scale), it computes the nearest integer value to arg (in floating-point format), rounding halfway cases away from zero. Scale and zero point must have same shape. They must be either scalar (per tensor) or 1-D tensor (per 'axis').

Version

This version of the operator has been available since version 1 of the 'com.microsoft' operator set.

Attributes

axis : int: The axis along which same quantization parameters are applied. It's optional. If it's not specified, it means per-tensor quantization and input 'x_scale' and 'x_zero_point' must be scalars. If it's specified, it means per 'axis' quantization and input 'x_scale' and 'x_zero_point' must be 1-D tensors.

Inputs

:
:
:

Outputs

:

Type Constraints

:
:

com.microsoft.ROIAlign

Region of Interest (RoI) align operation described in the Mask R-CNN paper. RoIAlign consumes an input tensor X and region of interests (rois) to apply pooling across each RoI; it produces a 4-D tensor of shape (num_rois, C, pooled_h, pooled_w).

RoIAlign is proposed to avoid the misalignment by removing
quantizations while converting from original image into feature
map and from feature map into RoI feature; in each ROI bin,
the value of the sampled locations are computed directly
through bilinear interpolation.

Version

This version of the operator has been available since version 1 of the 'com.microsoft' operator set.

Attributes

mode : string: The pooling method. Two modes are supported: 'avg' and 'max'. Default is 'avg'.
pooled_h : int: default 1; Pooled output Y's height.
pooled_w : int: default 1; Pooled output Y's width.
sampling_ratio : int: Number of sampling points in the interpolation grid used to compute the output value of each pooled output bin. If > 0, then exactly sampling_ratio x sampling_ratio grid points are used. If == 0, then an adaptive number of grid points are used (computed as ceil(roi_width / pooled_w), and likewise for height). Default is 0.
spatial_scale : float: Multiplicative spatial scale factor to translate ROI coordinates from their input spatial scale to the scale used when pooling, i.e., spatial scale of the input feature map X relative to the input image. E.g.; default is 1.0f.

Inputs

:
:

Outputs

:

Type Constraints

:

com.microsoft.Range

Creates a sequence of numbers that begins at start and extends by increments of delta up to but not including limit.

Version

This version of the operator has been available since version 1 of the 'com.microsoft' operator set.

Inputs (2 - 3)

:
:
(optional) :

Outputs

:

Type Constraints

:

com.microsoft.ReduceSumInteger

Computes the sum of the low-precision input tensor's element along the provided axes. The resulting tensor has the same rank as the input if keepdims equal 1. If keepdims equal 0, then the resulting tensor have the reduced dimension pruned. The above behavior is similar to numpy, with the exception that numpy default keepdims to False instead of True.

Version

This version of the operator has been available since version 1 of the 'com.microsoft' operator set.

Attributes

axes : list of ints (required): A list of integers, along which to reduce. The default is to reduce over all the dimensions of the input tensor.
keepdims : int (required): Keep the reduced dimension or not, default 1 mean keep reduced dimension.

Inputs

:

Outputs

:

Type Constraints

:
:

com.microsoft.SampleOp

Sample echo operator.

Version

This version of the operator has been available since version 1 of the 'com.microsoft' operator set.

Inputs

:

Outputs

:

Type Constraints

:

com.microsoft.Tokenizer

Tokenizer divides each string in X into a vector of strings along the last axis. Allowed input shapes are [C] and [N, C]. If the maximum number of tokens found per input string is D, the output shape would be [N, C, D] when input shape is [N, C]. Similarly, if input shape is [C] then the output should be [C, D]. Tokenizer has two different operation modes. The first mode is selected when "tokenexp" is not set and "separators" is set. If "tokenexp" is set and "separators" is not set, the second mode will be used. The first mode breaks each input string into tokens by removing separators.

Let's assume "separators" is [" "] and consider an example.
If input is

["Hello World", "I love computer science !"] whose shape is [2],

then the output would be

[["Hello", "World", padvalue, padvalue, padvalue], ["I", "love", "computer", "science", "!"]]

whose shape is [2, 5] because you can find at most 5 tokens per input string. Note that the input at most can have two axes, so 3-D and higher dimension are not supported.

For each input string, the second mode searches matches of "tokenexp" and each match will be a token in Y. The matching of "tokenexp" is conducted greedily (i.e., a match should be as long as possible). This operator searches for the first match starting from the beginning of the considered string, and then launches another search starting from the first remained character after the first matched token. If no match found, this operator will remove the first character from the remained string and do another search. This procedure will be repeated until reaching the end of the considered string.

Let's consider another example to illustrate the effect of setting "mark" to true.
If input is ["Hello", "World"],
then the corresponding output would be [0x02, "Hello", "World", 0x03].
This implies that if mark is true, [C]/[N, C] - input's output shape becomes [C, D+2]/[N, C, D+2].

Version

This version of the operator has been available since version 1 of the 'com.microsoft' operator set.

Attributes

mark : int (required): Boolean whether to mark the beginning/end character with start of text character (0x02)/end of text character (0x03).
mincharnum : int (required): Minimum number of characters allowed in the output. For example, if mincharnum is 2, tokens such as "A" and "B" would be ignored
pad_value : string (required): The string used to pad output tensors when the tokens extracted doesn't match the maximum number of tokens found. If start/end markers are needed, padding will appear outside the markers.
separators : list of strings: an optional list of strings (type: AttributeProto::STRINGS), each single string in this attribute is a separator. Two consecutive segments in X connected by a separator would be divided into two tokens. For example, if the input is "Hello World!" and this attribute contains only one space character, the corresponding output would be ["Hello", "World!"]. To achieve character-level tokenization, one should set the separators to [""], which contains only one empty string. If 'separators' is a L-element array, there will be L rounds of tokenization using one stop word. More specifically, in the first round, the first element in 'separators' is used to tokenize each string in the input. Then, the second element in 'separators' will be used to tokenize the resulted strings produced at the first round.
tokenexp : string: An optional string. Token's regular expression in basic POSIX format (http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html#tag_09_03). If set, tokenizer may produce tokens matching the specified pattern. Note that one and only of 'tokenexp' and 'separators' should be set.

Inputs

:

Outputs

:

Type Constraints

:

com.microsoft.WordConvEmbedding

The WordConvEmbedding takes in a batch of sequence words and embed each word to a vector.

Version

This version of the operator has been available since version 1 of the 'com.microsoft' operator set.

Attributes

char_embedding_size : int: Integer representing the embedding vector size for each char.If not provide, use the char embedding size of embedding vector.
conv_window_size : int: This operator applies convolution to word from left to right with window equal to conv_window_size and stride to 1.Take word 'example' for example, with conv_window_size equal to 2, conv is applied to [ex],[xa], [am], [mp]...If not provide, use the first dimension of conv kernal shape.
embedding_size : int: Integer representing the embedding vector size for each word.If not provide, use the fileter size of conv weight

Inputs

:
:
:
:

Outputs

:

Type Constraints

:
:

35 KiB Raw Blame History

Contrib Operator Schemas

com.microsoft

Version

Attributes

Inputs (3 - 14)

Outputs (0 - 3)

Type Constraints

Version

Attributes

Inputs (2 - 4)

Outputs

Type Constraints

Version

Attributes

Inputs

Outputs

Type Constraints

Version

Inputs

Outputs

Type Constraints

Version

Attributes

Inputs (2 - 3)

Outputs

Type Constraints

Version

Attributes

Inputs

Outputs

Type Constraints

Version

Inputs

Outputs

Type Constraints

Version

Inputs (2 - 4)

Outputs

Type Constraints

Version

Attributes

Inputs

Outputs

Type Constraints

Version

Attributes

Inputs

Outputs

Type Constraints

Version

Attributes

Inputs

Outputs (1 - 2)

Type Constraints

Version

Attributes

Inputs (8 - 9)

Outputs

Type Constraints

Version

Inputs

Outputs

Type Constraints

Version

Attributes

Inputs

Outputs

Type Constraints

Version

Attributes

Inputs

Outputs

Type Constraints

Version

Inputs (2 - 3)

Outputs

Type Constraints

Version

Attributes

35 KiB

Raw Blame History