onnxruntime/docs/ContribOperators.md

Contrib Operator Schemas

This file is automatically generated from the def files via this script. Do not modify directly and instead edit operator definitions.

• com.microsoft
  • com.microsoft.AttnLSTM
  • com.microsoft.ExpandDims
  • com.microsoft.FusedConv
  • com.microsoft.FusedGemm
  • com.microsoft.GatherND
  • com.microsoft.MaxpoolWithMask
  • com.microsoft.MurmurHash3
  • com.microsoft.Pad
  • com.microsoft.Range
  • com.microsoft.ReduceSumInteger
  • com.microsoft.SampleOp
  • com.microsoft.Tokenizer
  • com.microsoft.Unique
  • com.microsoft.WordConvEmbedding

com.microsoft

com.microsoft.AttnLSTM

Computes an one-layer RNN where its RNN Cell is an AttentionWrapper wrapped a LSTM Cell. The RNN layer contains following basic component: LSTM Cell, Bahdanau Attention Mechanism, AttentionWrapp.

Activation functions:

Relu(x)                - max(0, x)

Tanh(x)                - (1 - e^{-2x})/(1 + e^{-2x})

Sigmoid(x)             - 1/(1 + e^{-x})

(NOTE: Below are optional)

Affine(x)              - alpha*x + beta

LeakyRelu(x)           - x if x >= 0 else alpha * x

ThresholdedRelu(x)     - x if x >= alpha else 0

ScaledTanh(x)          - alpha*Tanh(beta*x)

HardSigmoid(x)         - min(max(alpha*x + beta, 0), 1)

Elu(x)                 - x if x >= 0 else alpha*(e^x - 1)

Softsign(x)            - x/(1 + |x|)

Softplus(x)            - log(1 + e^x)

Softmax(x)             - exp(x) / sum(exp(x))

Bahdanau Attention Mechanism: M - Memory tensor.

  `VALUES` - masked Memory by its real sequence length.

  `MW` - Memory layer weight.

  `KEYS` - Processed memory tensor by the memory layer.
           KEYS = M * MW

  `Query` - Query tensor, normally at specific time step in sequence.

  `QW` - Query layer weight in the attention mechanism

  `PQ` - processed query,  = `Query` * `QW`

  `V' - attention vector

  `ALIGN` - calculated alignment based on Query and KEYS
      ALIGN = softmax(reduce_sum(`V` * Tanh(`KEYS` + `PQ`)))

  `CONTEXT` - context based on `ALIGN` and `VALUES`
      CONTEXT = `ALIGN` * `VALUES`

LSTM Cell: X - input tensor concat with attention state in the attention wrapper

`i` - input gate

`o` - output gate

`f` - forget gate

`c` - cell gate

`t` - time step (t-1 means previous time step)

`W[iofc]` - W parameter weight matrix for input, output, forget, and cell gates

`R[iofc]` - R recurrence weight matrix for input, output, forget, and cell gates

`Wb[iofc]` - W bias vectors for input, output, forget, and cell gates

`Rb[iofc]` - R bias vectors for input, output, forget, and cell gates

`P[iof]`  - P peephole weight vector for input, output, and forget gates

`WB[iofc]` - W parameter weight matrix for backward input, output, forget, and cell gates

`RB[iofc]` - R recurrence weight matrix for backward input, output, forget, and cell gates

`WBb[iofc]` - W bias vectors for backward input, output, forget, and cell gates

`RBb[iofc]` - R bias vectors for backward input, output, forget, and cell gates

`PB[iof]`  - P peephole weight vector for backward input, output, and forget gates

`H` - Hidden state

`num_directions` - 2 if direction == bidirectional else 1

Equations (Default: f=Sigmoid, g=Tanh, h=Tanh):

  - it = f(Xt*(Wi^T) + Ht-1*(Ri^T) + Pi (.) Ct-1 + Wbi + Rbi)

  - ft = f(Xt*(Wf^T) + Ht-1*(Rf^T) + Pf (.) Ct-1 + Wbf + Rbf)

  - ct = g(Xt*(Wc^T) + Ht-1*(Rc^T) + Wbc + Rbc)

  - Ct = ft (.) Ct-1 + it (.) ct

  - ot = f(Xt*(Wo^T) + Ht-1*(Ro^T) + Po (.) Ct + Wbo + Rbo)

  - Ht = ot (.) h(Ct)

AttentionWrapp Notations: `lstm()' - wrapped inner cell. Ht, Ct = lstm(concat(Xt, ATTNt-1), Ct-1)

`am()` - attention mechanism the wrapper used.
         CONTEXTt, ALIGNt = am(Ht, ALIGNt-1)

`AW` - attention layer weights, optional.

`ATTN` - attention state, initial is zero. If `AW` provided, it is the output of the attention layer,
              ATTNt = concat(Ht, CONTEXTt) * AW
         otherwise,
              ATTNt = CONTEXTt

RNN layer output: Y - if needed is the sequence of Ht from lstm cell.

`Y_h` - is the last valid H from lstm cell.

`Y_c` - is the last valid C from lstm cell.

Version

This version of the operator has been available since version 1 of the 'com.microsoft' operator set.

Attributes

activation_alpha : list of floats

Optional scaling values used by some activation functions. The values are consumed in the order of activation functions, for example (f, g, h) in LSTM. Default values are the same as of corresponding ONNX operators.For example with LeakyRelu, the default alpha is 0.01.

activation_beta : list of floats

Optional scaling values used by some activation functions. The values are consumed in the order of activation functions, for example (f, g, h) in LSTM. Default values are the same as of corresponding ONNX operators.

activations : list of strings

A list of 3 (or 6 if bidirectional) activation functions for input, output, forget, cell, and hidden. The activation functions must be one of the activation functions specified above. Optional: See the equations for default if not specified.

clip : float

Cell clip threshold. Clipping bounds the elements of a tensor in the range of [-threshold, +threshold] and is applied to the input of activations. No clip if not specified.

direction : string

Specify if the RNN is forward, reverse, or bidirectional. Must be one of forward (default), reverse, or bidirectional.

hidden_size : int

Number of neurons in the hidden layer.

input_forget : int

Couple the input and forget gates if 1, default 0.

Inputs (3 - 14)

X : T

The input sequences packed (and potentially padded) into one 3-D tensor with the shape of `[seq_length, batch_size, input_size]`

W : T

The weight tensor for the gates. Concatenation of `W[iofc]` and `WB[iofc]` (if bidirectional) along dimension 0. The tensor has shape `[num_directions, 4*hidden_size, input_size]`.

R : T

The recurrence weight tensor. Concatenation of `R[iofc]` and `RB[iofc]` (if bidirectional) along dimension 0. This tensor has shape `[num_directions, 4*hidden_size, hidden_size]`.

B (optional) : T

The bias tensor for input gate. Concatenation of `[Wb[iofc], Rb[iofc]]`, and `[WBb[iofc], RBb[iofc]]` (if bidirectional) along dimension 0. This tensor has shape `[num_directions, 8*hidden_size]`. Optional: If not specified - assumed to be 0.

sequence_lens (optional) : T1

Optional tensor specifying lengths of the sequences in a batch. If not specified - assumed all sequences in the batch to have length `seq_length`. It has shape `[batch_size]`

initial_h (optional) : T

Optional initial value of the hidden. If not specified - assumed to be 0. It has shape `[num_directions, batch_size, hidden_size]`.

initial_c (optional) : T

Optional initial value of the cell. If not specified - assumed to be 0. It has shape `[num_directions, batch_size, hidden_size]`.

P (optional) : T

The weight tensor for peepholes. Concatenation of `P[iof]` and `PB[iof]` (if bidirectional) along dimension 0. It has shape `[num_directions, 3*hidde_size]`. Optional: If not specified - assumed to be 0.

QW (optional) : T

The weight tensor of the query layer in the attention mechanism. Should be of shape `[num_directions, am_query_depth(hidden_size of lstm), am_attn_size]`

MW (optional) : T

The weight tensor of the memory layer in the attention mechanism. Should be of shape `[num_directions, memory_depth, am_attn_size]`

V (optional) : T

The attention_v tensor in the attention mechanism. Should be of shape `[num_directions, am_attn_size]`

M (optional) : T

The sequence of the memory (input) for attention mechanism. Should be of `[batch_size, max_memory_step, memory_depth]`

memory_seq_lens (optional) : T1

The sequence length of the input memory for the attention mechanism. Should be of `[batch_size]`

AW (optional) : T

The weights of attention layer in the attention wrapper. If exists, should be of shape `[num_directions, memory_depth+hidden_size, aw_attn_size]. Please note that attention mechanism context depth is also memory_depth in the attention mechanism.`

Outputs (0 - 3)

Y (optional) : T

A tensor that concats all the intermediate output values of the hidden. It has shape `[seq_length, num_directions, batch_size, hidden_size]`

Y_h (optional) : T

The last output value of the hidden. It has shape `[num_directions, batch_size, hidden_size]`.

Y_c (optional) : T

The last output value of the cell. It has shape `[num_directions, batch_size, hidden_size]`.

Type Constraints

T : tensor(float), tensor(double)

Constrain input and output types to float tensors.

T1 : tensor(int32)

Constrain seq_lens to integral tensors.

com.microsoft.ExpandDims

ExpandDims echo operator.

Version

This version of the operator has been available since version 1 of the 'com.microsoft' operator set.

Inputs

X : T

input

axis : tensor(int32)

Specified axis to insert a dimension

Outputs

Y : T

output

Type Constraints

T : tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128)

Constrain to any tensor type. If the dtype attribute is not provided this must be a valid output type.

com.microsoft.FusedConv

The fused convolution operator schema is the same as Conv besides it includes an attribute activation.

Version

This version of the operator has been available since version 1 of the 'com.microsoft' operator set.

Attributes

activation : string

alpha : float

auto_pad : string

dilations : list of ints

group : int

kernel_shape : list of ints

pads : list of ints

strides : list of ints

Inputs (2 - 3)

X : T

W : T

B (optional) : T

Outputs

Y : T

Type Constraints

T : tensor(float16), tensor(float), tensor(double)

Constrain input and output types to float tensors

com.microsoft.FusedGemm

The FusedGemm operator schema is the same as Gemm besides it includes attributes activation and leaky_relu_alpha.

Version

This version of the operator has been available since version 1 of the 'com.microsoft' operator set.

Attributes

activation : string

alpha : float

Scalar multiplier for the product of input tensors A * B.

beta : float

Scalar multiplier for input tensor C.

leaky_relu_alpha : float

transA : int

Whether A should be transposed

transB : int

Whether B should be transposed

Inputs

A : T

Input tensor A. The shape of A should be (M, K) if transA is 0, or (K, M) if transA is non-zero.

B : T

Input tensor B. The shape of B should be (K, N) if transB is 0, or (N, K) if transB is non-zero.

C : T

Input tensor C. The shape of C should be unidirectional broadcastable to (M, N).

Outputs

Y : T

Output tensor of shape (M, N).

Type Constraints

T : tensor(float16), tensor(float), tensor(double), tensor(uint32), tensor(uint64), tensor(int32), tensor(int64)

Constrain input and output types to float/int tensors.

com.microsoft.GatherND

Given data tensor of rank r >= 1, and indices tensor of rank q >= 1, gather slices of data into an output tensor of rank q - 1 + r - indices[-1]. Example 1: data = 0,1],[2,3 indices = 0,0],[1,1 output = [0,3] Example 2: data = 0,1],[2,3 indices = 1],[0 output = 2,3],[0,1 Example 3: data = [0,1],[2,3,4,5],[6,7] indices = 0,1],[1,0 output = 2,3],[4,5 Example 4: data = [0,1],[2,3,4,5],[6,7] indices = [0,1,1,0] output = [2,3,4,5]

Version

This version of the operator has been available since version 1 of the 'com.microsoft' operator set.

Inputs

data : T

Tensor of rank r >= 1.

indices : Tind

Tensor of rank q >= 1.

Outputs

output : T

Tensor of rank q-1+r-indices[-1].

Type Constraints

T : tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128)

Constrain input and output types to any tensor type.

Tind : tensor(int32), tensor(int64)

Constrain indice type to int32 or int64

com.microsoft.MaxpoolWithMask

For internal use.

Version

This version of the operator has been available since version 1 of the 'com.microsoft' operator set.

Attributes

auto_pad : string

kernel_shape : list of ints

pads : list of ints

storage_order : int

strides : list of ints

Inputs

X : T

M : tensor(int32)

mask

Outputs

Y : T

Type Constraints

T : tensor(float)

Constrain input0 and output types to float tensors

com.microsoft.MurmurHash3

The underlying implementation is MurmurHash3_x86_32 generating low latency 32bits hash suitable for implementing lookup tables, Bloom filters, count min sketch or feature hashing.

Version

This version of the operator has been available since version 1 of the 'com.microsoft' operator set.

Attributes

positive : int

If value is 1, output type is uint32_t, else int32_t. Default value is 1.

seed : int

Seed for the hashing algorithm, unsigned 32-bit integer, default to 0.

Inputs

X : T1

An input tensor to hash.

Outputs

Y : T2

32-bit hash value.

Type Constraints

T1 : tensor(uint32), tensor(int32), tensor(string)

Constrain input type to unsigned or signed 32-bit integer tensor, or string tensor. It should be utf-8 encoded if using unicode.

T2 : tensor(uint32), tensor(int32)

Constrain output type to unsigned and signed 32-bit integer tensor.

com.microsoft.Pad

Given data tensor, pads, mode, and value. Example: Insert 0 pads to the beginning of the second dimension. data = [ [1.0, 1.2], [2.3, 3.4], [4.5, 5.7], ] pads = [0, 2, 0, 0] output = [ [ [0.0, 0.0, 1.0, 1.2], [0.0, 0.0, 2.3, 3.4], [0.0, 0.0, 4.5, 5.7], ], ]

Version

This version of the operator has been available since version 1 of the 'com.microsoft' operator set.

Attributes

mode : string

Three modes: `constant`(default) - pads with a given constant value, `reflect` - pads with the reflection of the vector mirrored on the first and last values of the vector along each axis, `edge` - pads with the edge values of array

Inputs (2 - 3)

data : T

Input tensor.

pads : tensor(int64)

Tensor of integers indicating the number of padding elements to add or remove (if negative) at the beginning and end of each axis. For 2D input tensor, it is the number of pixels. `pads` should be a 1D tensor of shape [2 * input_rank] or a 2D tensor of shape [1, 2 * input_rank]. `pads` format (1D example) should be as follow [x1_begin, x2_begin,...,x1_end, x2_end,...], where xi_begin is the number of pixels added at the beginning of axis `i` and xi_end, the number of pixels added at the end of axis `i`.

value (optional) : T

(Optional) A scalar or rank 1 tensor containing a single value to be filled if the mode chosen is `constant` (by default it is 0.0).

Outputs

output : T

Tensor after padding.

Type Constraints

T : tensor(float16), tensor(float), tensor(double)

Constrain input and output types to float tensors.

com.microsoft.Range

Creates a sequence of numbers that begins at start and extends by increments of delta up to but not including limit.

Version

This version of the operator has been available since version 1 of the 'com.microsoft' operator set.

Inputs (2 - 3)

start : T

Tensor(scalar, or dims=[1]). First entry in the range.

limit : T

Tensor(scalar, or dims=[1]). Upper limit of sequence, exclusive.

delta (optional) : T

Tensor(scalar, or dims=[1]). Number that increments start. Defaults to 1.

Outputs

Y : T

1-D Tensor of the range.

Type Constraints

T : tensor(float), tensor(double), tensor(int16), tensor(int32), tensor(int64)

Constrain input and output types.

com.microsoft.ReduceSumInteger

Computes the sum of the low-precision input tensor's element along the provided axes. The resulting tensor has the same rank as the input if keepdims equal 1. If keepdims equal 0, then the resulting tensor have the reduced dimension pruned. The above behavior is similar to numpy, with the exception that numpy default keepdims to False instead of True.

Version

This version of the operator has been available since version 1 of the 'com.microsoft' operator set.

Attributes

axes : list of ints (required)

A list of integers, along which to reduce. The default is to reduce over all the dimensions of the input tensor.

keepdims : int (required)

Keep the reduced dimension or not, default 1 mean keep reduced dimension.

Inputs

data : T1

An input tensor.

Outputs

reduced : T2

Reduced output tensor.

Type Constraints

T1 : tensor(int8), tensor(uint8)

Constrain input type to 8-bit integer tensor.

T2 : tensor(int32), tensor(uint32)

Constrain output data type to 32-bit integer tensor.T2 must be tensor(uint32) when T1 is tensor(uint8),or must be tensor(int32) when T1 is tensor(int8).

com.microsoft.SampleOp

Sample echo operator.

Version

This version of the operator has been available since version 1 of the 'com.microsoft' operator set.

Inputs

X : T

input

Outputs

Y : T

output

Type Constraints

T : tensor(uint32), tensor(uint64), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double)

Constrain to any tensor type. If the dtype attribute is not provided this must be a valid output type.

com.microsoft.Tokenizer

Tokenizer divides each string in X into a vector of strings along the last axis. Allowed input shapes are [C] and [N, C]. If the maximum number of tokens found per input string is D, the output shape would be [N, C, D] when input shape is [N, C]. Similarly, if input shape is [C] then the output should be [C, D]. Tokenizer has two different operation modes. The first mode is selected when "tokenexp" is not set and "separators" is set. If "tokenexp" is set and "separators" is not set, the second mode will be used. The first mode breaks each input string into tokens by matching and removing separators. "separators" is a list of strings which are regular expressions. "tokenexp" is a single regular expression.

Let's assume "separators" is [" "] and consider an example.
If input is

["Hello World", "I love computer science !"] whose shape is [2],

then the output would be

[["Hello", "World", padvalue, padvalue, padvalue], ["I", "love", "computer", "science", "!"]]

whose shape is [2, 5] because you can find at most 5 tokens per input string. Note that the input at most can have two axes, so 3-D and higher dimension are not supported.

If "separators" contains a single empty string, the Tokenizer will enter into character tokenezation mode. This means all strings will be broken part into individual characters.

For each input string, the second mode searches matches of "tokenexp" and each match will be a token in Y. The matching of "tokenexp" is conducted greedily (i.e., a match should be as long as possible). This operator searches for the first match starting from the beginning of the considered string, and then launches another search starting from the first remained character after the first matched token. If no match found, this operator will remove the first character from the remained string and do another search. This procedure will be repeated until reaching the end of the considered string.

Let's consider another example to illustrate the effect of setting "mark" to true.
If input is ["Hello", "World"],
then the corresponding output would be [0x02, "Hello", "World", 0x03].
This implies that if mark is true, [C]/[N, C] - input's output shape becomes [C, D+2]/[N, C, D+2].

If tokenizer removes the entire content of [C]-input, it will produce . I.e. the output shape should be [C][0] or [N][C][0] if input shape was [N][C].

If the tokenizer receives empty input of [0] then the output is [0] if empty input of [N, 0] then [N, 0].

Version

This version of the operator has been available since version 1 of the 'com.microsoft' operator set.

Attributes

mark : int (required)

Boolean whether to mark the beginning/end character with start of text character (0x02)/end of text character (0x03).

mincharnum : int (required)

Minimum number of characters allowed in the output. For example, if mincharnum is 2, tokens such as "A" and "B" would be ignored

pad_value : string (required)

The string used to pad output tensors when the tokens extracted doesn't match the maximum number of tokens found. If start/end markers are needed, padding will appear outside the markers.

separators : list of strings

an optional list of strings attribute that contains a list of separators - regular expressions to match separators Two consecutive segments in X connected by a separator would be divided into two tokens. For example, if the input is "Hello World!" and this attribute contains only one space character, the corresponding output would be ["Hello", "World!"]. To achieve character-level tokenization, one should set the 'separators' to [""], which contains an empty string.

tokenexp : string

An optional string. Token's regular expression in basic POSIX format (http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html#tag_09_03). If set, tokenizer may produce tokens matching the specified pattern. Note that one and only of 'tokenexp' and 'separators' should be set.

Inputs

X : T

Strings to tokenize

Outputs

Y : T

Tokenized strings

Type Constraints

T : tensor(string)

Input/Output is a string tensor

com.microsoft.Unique

Finds all the unique values (deduped list) present in the given input tensor. This operator returns 3 outputs. The first output tensor 'uniques' contains all of the unique elements of the input, sorted in the same order that they occur in the input. The second output tensor 'idx' is the same size as the input and it contains the index of each value of the input in 'uniques'. The third output tensor 'counts' contains the count of each element of 'uniques' in the input. Example: input_x = [2, 1, 1, 3, 4, 3] output_uniques = [2, 1, 3, 4] output_idx = [0, 1, 1, 2, 3, 2] output_counts = [1, 2, 2, 1]

Version

This version of the operator has been available since version 1 of the 'com.microsoft' operator set.

Inputs

x : T

A 1-D input tensor that is to be processed.

Outputs

y : T

A 1-D tensor of the same type as 'x' containing all the unique values in 'x' sorted in the same order that they occur in the input 'x'

idx : tensor(int64)

A 1-D INT64 tensor of the same size as 'x' containing the indices for each value in 'x' in the output 'uniques'

counts : tensor(int64)

A 1-D INT64 tensor containing the the count of each element of 'uniques' in the input 'x'

Type Constraints

T : tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double), tensor(string), tensor(bool), tensor(complex64), tensor(complex128)

Input can be of any tensor type.

com.microsoft.WordConvEmbedding

The WordConvEmbedding takes in a batch of sequence words and embed each word to a vector.

Version

This version of the operator has been available since version 1 of the 'com.microsoft' operator set.

Attributes

char_embedding_size : int

Integer representing the embedding vector size for each char.If not provide, use the char embedding size of embedding vector.

conv_window_size : int

This operator applies convolution to word from left to right with window equal to conv_window_size and stride to 1.Take word 'example' for example, with conv_window_size equal to 2, conv is applied to [ex],[xa], [am], [mp]...If not provide, use the first dimension of conv kernal shape.

embedding_size : int

Integer representing the embedding vector size for each word.If not provide, use the fileter size of conv weight

Inputs

Sequence : T

Specify batchs of sequence words to embedding

W : T1

Specify weights of conv

B : T1

Specify bias of conv

C : T1

Specify embedding vector of char

Outputs

Y : T1

output

Type Constraints

T : tensor(int32)

Constrain to tensor(int32).

T1 : tensor(float)

Constrain to tensor(float).