2023-05-20 19:20:41 +00:00
## Operators Support Table
The following table shows ONNX
operators and the supported opset domain/versions in WebGPU EP by ONNX Runtime Web. For example,
`4-6, 8+` means ONNX Runtime Web currently support opset version 4 to 6, 8 and above.
*This file is automatically generated from the
def files via [this script ](../script/generate-webgpu-operator-md.ts ).
Do not modify directly.*
| Operator | Opset | Comments |
|:--------:|:-------------:|-----|
| Abs | ai.onnx(6-12,13+) | |
| Acos | ai.onnx(7+) | |
| Acosh | ai.onnx(9+) | |
| Add | ai.onnx(7-12,13,14+) | |
2023-08-03 01:16:19 +00:00
| ArgMax | ai.onnx(1-10,11-12,13+) | |
| ArgMin | ai.onnx(1-10,11-12,13+) | |
2023-05-20 19:20:41 +00:00
| Asin | ai.onnx(7+) | |
| Asinh | ai.onnx(9+) | |
| Atan | ai.onnx(7+) | |
| Atanh | ai.onnx(9+) | |
2023-11-17 20:23:52 +00:00
| Attention | com.microsoft(1+) | need implementing mask and past/present |
2023-11-03 16:04:28 +00:00
| AveragePool | ai.onnx(7-9,10,11+); com.ms.internal.nhwc(7-9,10,11+) | need perf optimization; need implementing activation |
2023-11-22 23:58:06 +00:00
| BatchNormalization | ai.onnx(7-8,9-13,14,15+); com.ms.internal.nhwc(7-8,9-13,14,15+) | |
2023-10-03 19:20:20 +00:00
| BiasAdd | com.microsoft(1+) | |
| BiasSplitGelu | com.microsoft(1+) | |
2023-08-19 06:51:03 +00:00
| Cast | ai.onnx(6-8,9-12,13-18,19+) | |
2023-05-20 19:20:41 +00:00
| Ceil | ai.onnx(6-12,13+) | |
| Clip | ai.onnx(6-10,11,12,13+) | |
2023-07-05 18:59:45 +00:00
| Concat | ai.onnx(1-3,4-10,11-12,13+) | |
2023-11-03 16:04:28 +00:00
| Conv | ai.onnx(1-10,11+); com.ms.internal.nhwc(1-10,11+) | need perf optimization; conv3d is not supported; need implementing activation |
| ConvTranspose | ai.onnx(1-10,11+); com.ms.internal.nhwc(1-10,11+) | need perf optimization; ConvTranspose3d is not supported; need implementing activation |
2023-05-20 19:20:41 +00:00
| Cos | ai.onnx(7+) | |
| Cosh | ai.onnx(9+) | |
2023-12-05 15:51:53 +00:00
| CumSum | ai.onnx(11-13,14+) | |
2024-04-10 19:13:46 +00:00
| DepthToSpace | ai.onnx(11-12,13+); com.ms.internal.nhwc(11-12,13+) | |
2023-05-20 19:20:41 +00:00
| Div | ai.onnx(7-12,13,14+) | |
2023-09-11 22:57:15 +00:00
| Einsum | ai.onnx(12+) | |
2023-05-20 19:20:41 +00:00
| Elu | ai.onnx(6+) | |
2023-08-28 02:50:17 +00:00
| Equal | ai.onnx(7-10,11-12,13-18,19+) | |
2023-05-20 19:20:41 +00:00
| Erf | ai.onnx(9-12,13+) | |
| Exp | ai.onnx(6-12,13+) | |
2023-07-11 16:38:16 +00:00
| Expand | ai.onnx(8-12,13+) | |
2024-02-06 17:07:31 +00:00
| FastGelu | com.microsoft(1+) | |
2023-07-27 19:50:45 +00:00
| Flatten | ai.onnx(1-8,9-10,11-12,13+) | |
2023-05-20 19:20:41 +00:00
| Floor | ai.onnx(6-12,13+) | |
2023-11-01 22:34:51 +00:00
| FusedConv | com.microsoft(1+) | |
2023-08-03 21:09:37 +00:00
| Gather | ai.onnx(1-10,11-12,13+) | |
2023-08-28 16:55:25 +00:00
| GatherElements | ai.onnx(11-12,13+) | |
2023-07-31 16:18:58 +00:00
| Gelu | com.microsoft(1+) | |
2023-08-23 01:13:20 +00:00
| Gemm | ai.onnx(7-8,9-10,11-12,13+) | |
2023-05-20 19:20:41 +00:00
| GlobalAveragePool | ai.onnx(1+); com.ms.internal.nhwc(1+) | |
| GlobalMaxPool | ai.onnx(1+); com.ms.internal.nhwc(1+) | |
2023-08-25 19:11:25 +00:00
| Greater | ai.onnx(7-8,9-12,13+) | |
2023-09-08 00:41:16 +00:00
| GreaterOrEqual | ai.onnx(12-15,16+) | |
2024-05-13 16:43:37 +00:00
| GroupQueryAttention | com.microsoft(1+) | |
2024-01-22 23:53:26 +00:00
| HardSigmoid | ai.onnx(6+) | |
2023-09-19 19:20:18 +00:00
| If | ai.onnx(1-10,11-12,13-18,19+) | |
2023-08-08 16:09:37 +00:00
| InstanceNormalization | ai.onnx(6+); com.ms.internal.nhwc(6+) | |
2024-06-27 17:20:48 +00:00
| LayerNormalization | ai.onnx(1-16,17+) | |
2023-05-20 19:20:41 +00:00
| LeakyRelu | ai.onnx(6-15,16+) | |
2023-08-25 19:11:25 +00:00
| Less | ai.onnx(7-8,9-12,13+) | |
2023-09-08 00:41:16 +00:00
| LessOrEqual | ai.onnx(12-15,16+) | |
2023-08-15 01:04:12 +00:00
| Log | ai.onnx(6-12,13+) | |
2023-05-20 19:20:41 +00:00
| MatMul | ai.onnx(1-12,13+) | |
2024-02-17 17:19:17 +00:00
| MatMulNBits | com.microsoft(1+) | |
2023-11-03 16:04:28 +00:00
| MaxPool | ai.onnx(1-7,8-9,10,11,12+); com.ms.internal.nhwc(1-7,8-9,10,11,12+) | need perf optimization; need implementing activation |
2023-05-20 19:20:41 +00:00
| MemcpyFromHost | ai.onnx(1+) | |
| MemcpyToHost | ai.onnx(1+) | |
| Mul | ai.onnx(7-12,13,14+) | |
2023-11-17 20:23:52 +00:00
| MultiHeadAttention | com.microsoft(1+) | need implementing mask and past/present |
2023-05-20 19:20:41 +00:00
| Neg | ai.onnx(6-12,13+) | |
2023-08-28 02:50:17 +00:00
| Not | ai.onnx(1+) | |
2023-09-14 20:14:11 +00:00
| Pad | ai.onnx(2-10,11-12,13-17,18,19+) | |
2023-05-20 19:20:41 +00:00
| Pow | ai.onnx(7-11,12,13-14,15+) | |
2024-06-06 15:21:33 +00:00
| QuickGelu | com.microsoft(1+) | |
2023-09-30 09:05:32 +00:00
| Range | ai.onnx(11+) | |
2023-05-20 19:20:41 +00:00
| Reciprocal | ai.onnx(6-12,13+) | |
2023-06-12 14:46:27 +00:00
| ReduceL1 | ai.onnx(1-10,11-12,13-17,18+) | |
| ReduceL2 | ai.onnx(1-10,11-12,13-17,18+) | |
| ReduceLogSum | ai.onnx(1-10,11-12,13-17,18+) | |
| ReduceLogSumExp | ai.onnx(1-10,11-12,13-17,18+) | |
| ReduceMax | ai.onnx(1-10,11,12,13-17,18+) | |
| ReduceMean | ai.onnx(1-10,11-12,13-17,18+) | |
| ReduceMin | ai.onnx(1-10,11,12,13-17,18+) | |
| ReduceProd | ai.onnx(1-10,11-12,13-17,18+) | |
| ReduceSum | ai.onnx(1-10,11-12,13+) | |
| ReduceSumSquare | ai.onnx(1-10,11-12,13-17,18+) | |
2023-05-20 19:20:41 +00:00
| Relu | ai.onnx(6-12,13,14+) | |
| Reshape | ai.onnx(5-12,13,14+) | no GPU kernel |
2023-11-03 16:04:28 +00:00
| Resize | ai.onnx(10,11-12,13-17,18,19+); com.ms.internal.nhwc(10,11-12,13-17,18,19+) | CoordinateTransformMode align_corners is not supported with downsampling |
2024-04-08 16:11:26 +00:00
| RotaryEmbedding | com.microsoft(1+) | |
2023-05-20 19:20:41 +00:00
| Shape | ai.onnx(1-12,13-14,15+) | no GPU kernel; an ORT warning is generated - need to fix |
| Sigmoid | ai.onnx(6-12,13+) | |
2024-04-11 21:08:50 +00:00
| SimplifiedLayerNormalization | ai.onnx(1+) | |
2023-05-20 19:20:41 +00:00
| Sin | ai.onnx(7+) | |
| Sinh | ai.onnx(9+) | |
2023-08-18 16:59:03 +00:00
| SkipLayerNormalization | com.microsoft(1+) | |
2024-04-11 21:08:50 +00:00
| SkipSimplifiedLayerNormalization | com.microsoft(1+) | |
2023-07-25 21:19:20 +00:00
| Slice | ai.onnx(1-9,10,11-12,13+) | |
2023-08-03 01:16:19 +00:00
| Softmax | ai.onnx(1-10,11-12,13+) | |
2023-07-07 19:16:10 +00:00
| Split | ai.onnx(1,2-10,11-12,13-17,18+) | |
2023-05-20 19:20:41 +00:00
| Sqrt | ai.onnx(6-12,13+) | |
[JS/WebGPU] Squeeze operator implementation (#16024)
### Description
This PR adds an implementation of the `Squeeze` operator to WebGPU JSEP.
The implementation follows the [operator
schema](https://github.com/onnx/onnx/blob/main/docs/Operators.md#Squeeze)
and allows one or two inputs.
### How was it tested
1. I created two models. Without `axes`:
```Python
import onnx.helper
node = onnx.helper.make_node(
"Squeeze",
inputs=["T"],
outputs=["y"],
)
graph = onnx.helper.make_graph([node], "test", [onnx.helper.make_tensor_value_info("T", 1, [3, 1, 4, 5])],
[onnx.helper.make_tensor_value_info("y", 1, [3, 4, 5])])
onnx.save(onnx.helper.make_model(graph), "squeeze.onnx")
```
And with `axes`:
```Python
import onnx.helper
node = onnx.helper.make_node(
"Squeeze",
inputs=["T", "axes"],
outputs=["y"],
)
graph = onnx.helper.make_graph([node], "test", [onnx.helper.make_tensor_value_info("T", 1, [3, 1, 4, 5]), onnx.helper.make_tensor_value_info("axes", 7, [1])], [onnx.helper.make_tensor_value_info("y", 1, [3, 4, 5])])
onnx.save(onnx.helper.make_model(graph), "squeeze-dim.onnx")
```
2. I compiled the runtime using @fs-eire's
[instructions](https://gist.github.com/fs-eire/a55b2c7e10a6864b9602c279b8b75dce).
3. I ran the test models in the browser using this minimal setup:
```HTML
<html>
<script src=".\dist\ort.webgpu.min.js"></script>
<script>
async function run() {
const session = await ort.InferenceSession.create('squeeze-dim.onnx', {executionProviders: ['webgpu']});
console.log(session);
const input = new ort.Tensor('float32', new Float32Array(60), [3, 1, 4, 5]);
const dim = new ort.Tensor('int64', [-3n], [1]);
const output = await session.run({ "T": input, "axes": dim });
console.log(output);
}
run();
</script>
</html>
```
### Motivation and Context
Improve operator coverage for WebGPU JSEP.
2023-05-26 22:53:05 +00:00
| Squeeze | ai.onnx(1-10,11-12,13+) | |
2023-05-20 19:20:41 +00:00
| Sub | ai.onnx(7-12,13,14+) | |
| Tan | ai.onnx(7+) | |
2023-08-12 18:43:39 +00:00
| Tanh | ai.onnx(6-12,13+) | |
2023-05-20 19:20:41 +00:00
| ThresholdedRelu | ai.onnx(10+) | |
2023-08-18 17:07:21 +00:00
| Tile | ai.onnx(6-12,13+) | |
2023-05-20 19:20:41 +00:00
| Transpose | ai.onnx(1-12,13+) | need perf optimization |
[JS/WebGPU] Unsqueeze operator implementation (#16138)
### Description
This PR adds an implementation of the Squeeze operator to WebGPU JSEP.
The implementation follows the [operator
schema](https://github.com/onnx/onnx/blob/main/docs/Operators.md#Unsqueeze).
To implement the `Unsqueeze` operator in the same fashion as the
`Squeeze`, I added the `ComputeOutputShape()` method to the
`UnsqueezeBase` class and made some slight modifications. Please let me
know if it is a bad idea and if I should move this method to the JS
implementation.
I also uncommented test case lines in the `suite-test-list.jsonc` file
for both Squeeze and Unsqueeze operators following @hariharans29's
[comment](https://github.com/microsoft/onnxruntime/pull/16024#issuecomment-1565113633).
### How was it tested
1. I created a model with only one operator:
```Python
import onnx.helper
node = onnx.helper.make_node(
"Unsqueeze",
inputs=["T", "axes"],
outputs=["y"],
)
graph = onnx.helper.make_graph([node], "test", [onnx.helper.make_tensor_value_info("T", 1, [3, 4, 5]), onnx.helper.make_tensor_value_info("axes", 7, [2])], [onnx.helper.make_tensor_value_info("y", 1, [3, 1, 4, 5, 1])])
onnx.save(onnx.helper.make_model(graph), "unsqueeze.onnx")
```
2. I compiled the runtime using @fs-eire's
[instructions](https://gist.github.com/fs-eire/a55b2c7e10a6864b9602c279b8b75dce).
3. I ran the test models in the browser using this minimal setup:
```HTML
<html>
<script src=".\dist\ort.webgpu.min.js"></script>
<script>
async function run() {
const session = await ort.InferenceSession.create('unsqueeze.onnx', {executionProviders: ['webgpu']});
console.log(session);
const input = new ort.Tensor('float32', new Float32Array(60), [3, 4, 5]);
const dim = new ort.Tensor('int64', [1n, 4n], [2]);
const output = await session.run({ "T": input, "axes": dim });
console.log(output);
}
run();
</script>
</html>
```
### Motivation and Context
Improve operator coverage for WebGPU JSEP.
2023-06-01 19:23:02 +00:00
| Unsqueeze | ai.onnx(1-10,11-12,13+) | |
2023-10-03 21:28:21 +00:00
| Where | ai.onnx(9-15,16+) | |
