mirror of
https://github.com/saymrwulf/onnxruntime.git
synced 2026-07-30 20:18:08 +00:00
[Xnnpack] update doc (#13624)
### Description - iOS support - MatMul && GEMM - Resize - ConvTranspose A deployment [link](https://wejoncy.github.io/onnxruntime/) for showcase. ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. -->
This commit is contained in:
JiCheng
GitHub
parent
bc6cafbae1
commit
88928251eb
3 changed files with 35 additions and 7 deletions
19
docs/build/eps.md
vendored
19
docs/build/eps.md
vendored
|
|
@ -723,13 +723,14 @@ Once you have all the necessary components setup, follow the instructions to [cr
|
|||
|
||||
## XNNPACK
|
||||
|
||||
Usage of XNNPACK on Android/Windows/Linux platforms is via the XNNPACK EP.
|
||||
Usage of XNNPACK on Android/iOS/Windows/Linux platforms is via the XNNPACK EP.
|
||||
|
||||
See the [XNNPACK Execution Provider](../execution-providers/Xnnpack-ExecutionProvider.md) documentation for more details.
|
||||
|
||||
The pre-built ONNX Runtime package([`onnxruntime-android`](https://mvnrepository.com/artifact/com.microsoft.onnxruntime/onnxruntime-android)) for Android includes the XNNPACK EP.
|
||||
|
||||
We don't support XNNPACK on iOS currently.
|
||||
The pre-built ONNX Runtime Mobile package for iOS, `onnxruntime-c` and `onnxruntime-objc` in [CocoaPods](https://cocoapods.org/), includes the XNNPACK EP. (Package `onnxruntime-objc` with XNNPACK will be available since 1.14.)
|
||||
|
||||
|
||||
If performing a custom build of ONNX Runtime, support for the XNNPACK EP must be enabled when building.
|
||||
|
||||
|
|
@ -761,6 +762,20 @@ Linux example:
|
|||
```bash
|
||||
./build.sh --cmake_generator "Ninja" --android --android_sdk_path /Android --android_ndk_path /Android/ndk/21.1.6352462/ --android_abi arm64-v8a --android_api 29 --use_xnnpack
|
||||
```
|
||||
### Build for iOS (available since 1.14)
|
||||
A Mac machine is required to build package for iOS. Please follow this [guide](./ios.md) to set up environment firstly.
|
||||
#### Create a minimal build with XNNPACK EP support
|
||||
|
||||
Once you have all the necessary components setup, follow the instructions to [create the custom build](./custom.md), with the following changes:
|
||||
|
||||
* Replace `--minimal_build` with `--minimal_build extended` to enable support for execution providers that dynamically create kernels at runtime, which is required by the XNNPACK EP.
|
||||
* Add `--use_xnnpack` to include the XNNPACK EP in the build
|
||||
|
||||
```dos
|
||||
<ONNX Runtime repository root>./build.sh --config <Release|Debug|RelWithDebInfo|MinSizeRel> --use_xcode \
|
||||
--ios --ios_sysroot iphoneos --osx_arch arm64 --apple_deploy_target <minimal iOS version> --use_xnnpack --minimal_build extended --disable_ml_ops --disable_exceptions --build_shared_lib --skip_tests --include_ops_by_config <config file from model conversion>
|
||||
```
|
||||
|
||||
### Build for Windows
|
||||
```dos
|
||||
<ONNX Runtime repository root>.\build.bat --config <Release|Debug|RelWithDebInfo> --use_xnnpack
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@ nav_order: 9
|
|||
|
||||
# XNNPACK Execution Provider
|
||||
|
||||
Accelerate ONNX models on Android devices and WebAssembly with ONNX Runtime and the XNNPACK execution provider. [(XNNPACK)](https://github.com/google/XNNPACK) is a highly optimized library of floating-point neural network inference operators for ARM, WebAssembly, and x86 platforms.
|
||||
Accelerate ONNX models on Android/iOS devices and WebAssembly with ONNX Runtime and the XNNPACK execution provider. [(XNNPACK)](https://github.com/google/XNNPACK) is a highly optimized library of floating-point neural network inference operators for ARM, WebAssembly, and x86 platforms.
|
||||
|
||||
## Contents
|
||||
{: .no_toc }
|
||||
|
|
@ -23,12 +23,14 @@ Accelerate ONNX models on Android devices and WebAssembly with ONNX Runtime and
|
|||
Pre-built packages of ONNX Runtime([`onnxruntime-android`](https://mvnrepository.com/artifact/com.microsoft.onnxruntime/onnxruntime-android)) with XNNPACK EP for Android are published on Maven.
|
||||
See [here](../install/index.md#install-on-android) for installation instructions.
|
||||
|
||||
There is no pre-built package for iOS yet. We will update it once it is ready.
|
||||
Pre-built binaries(`onnxruntime-objc` and `onnxruntime-c`) of ONNX Runtime with XNNPACK EP for iOS are published to CocoaPods.
|
||||
See [here](../install/index.md#install-on-ios) for installation instructions.
|
||||
|
||||
## Build
|
||||
|
||||
Please see the [Build page](../build/eps.md#xnnpack) for instructions on building a package that includes the XNNPACK EP.
|
||||
|
||||
Support build for Android/Windows/Linux
|
||||
Support build for Android/iOS/Windows/Linux
|
||||
|
||||
## Usage
|
||||
|
||||
|
|
@ -75,12 +77,17 @@ The number of threads to use for the XNNPACK EP's internal intra-op thread-pool.
|
|||
## Supported ops
|
||||
Following ops are supported by the XNNPACK Execution Provider,
|
||||
|
||||
|Operator|Note|
|
||||
|--------|------|
|
||||
|Operator|Note||
|
||||
|--------|------|-----|
|
||||
|ai.onnx:AveragePool|Only 2D Pool is supported.|
|
||||
|ai.onnx:Conv|Only 2D Conv is supported.<br/>Weights and bias should be constant.|
|
||||
|ai.onnx:ConvTranspose|Only 2D ConvTranspose is supported.<br/>Weights and bias should be constant.|since 1.14|
|
||||
|ai.onnx:MaxPool|Only 2D Pool is supported.|
|
||||
|ai.onnx:Softmax|all opset below 13 is supported, only support opset 13 when AXIS is the last dimension|
|
||||
|ai.onnx:QLinearConv|Only 2D Conv is supported.<br/>Weights and bias should be constant.<br/>All quantization scales and zero points should be constant.|
|
||||
|ai.onnx:Resize|2D/4D Resize in `Bilinear mode` are supported|since 1.14|
|
||||
|ai.onnx:Gemm|Only 2D Op is supported|since 1.14|
|
||||
|ai.onnx:Matmul|Only 2D Op is supported|since 1.14|
|
||||
|com.microsoft:QLinearAveragePool|Only 2D Pool is supported.<br/>All quantization scales and zero points should be constant.|
|
||||
|com.microsoft:QLinearSoftmax|All quantization scales and zero points should be constant.|
|
||||
|com.microsoft:QLinearConvTranspose|All quantization scales and zero points should be constant.|since 1.14|
|
||||
|
|
|
|||
|
|
@ -1169,6 +1169,12 @@ var validCombos = {
|
|||
"Follow build instructions from <a href='https://aka.ms/build-ort-snpe' target='_blank'>here</a>",
|
||||
|
||||
//Xnnpack
|
||||
"ios,C-API,ARM64,XNNPACK":
|
||||
"Add 'onnxruntime-c' using CocoaPods and refer to the <a href='https://onnxruntime.ai/docs/tutorials/mobile/' target='_blank'>mobile deployment guide</a> or Follow build instructions from <a href='https://aka.ms/build-ort-xnnpack' target='_blank'>here</a>",
|
||||
|
||||
"ios,objectivec,ARM64,XNNPACK":
|
||||
"Add 'onnxruntime-objc' using CocoaPods and refer to the <a href='https://onnxruntime.ai/docs/tutorials/mobile/' target='_blank'>mobile deployment guide</a>",
|
||||
|
||||
"android,C-API,ARM64,XNNPACK":
|
||||
"Follow build instructions from <a href='https://aka.ms/build-ort-xnnpack' target='_blank'>here</a>",
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue