From 88928251eb844ff81caea7c1ae3536fd3882b806 Mon Sep 17 00:00:00 2001 From: JiCheng Date: Fri, 20 Jan 2023 11:14:48 +0800 Subject: [PATCH] [Xnnpack] update doc (#13624) ### Description - iOS support - MatMul && GEMM - Resize - ConvTranspose A deployment [link](https://wejoncy.github.io/onnxruntime/) for showcase. ### Motivation and Context --- docs/build/eps.md | 19 +++++++++++++++++-- .../Xnnpack-ExecutionProvider.md | 17 ++++++++++++----- js/script.js | 6 ++++++ 3 files changed, 35 insertions(+), 7 deletions(-) diff --git a/docs/build/eps.md b/docs/build/eps.md index da36d5fadc..f429b74515 100644 --- a/docs/build/eps.md +++ b/docs/build/eps.md @@ -723,13 +723,14 @@ Once you have all the necessary components setup, follow the instructions to [cr ## XNNPACK -Usage of XNNPACK on Android/Windows/Linux platforms is via the XNNPACK EP. +Usage of XNNPACK on Android/iOS/Windows/Linux platforms is via the XNNPACK EP. See the [XNNPACK Execution Provider](../execution-providers/Xnnpack-ExecutionProvider.md) documentation for more details. The pre-built ONNX Runtime package([`onnxruntime-android`](https://mvnrepository.com/artifact/com.microsoft.onnxruntime/onnxruntime-android)) for Android includes the XNNPACK EP. -We don't support XNNPACK on iOS currently. +The pre-built ONNX Runtime Mobile package for iOS, `onnxruntime-c` and `onnxruntime-objc` in [CocoaPods](https://cocoapods.org/), includes the XNNPACK EP. (Package `onnxruntime-objc` with XNNPACK will be available since 1.14.) + If performing a custom build of ONNX Runtime, support for the XNNPACK EP must be enabled when building. @@ -761,6 +762,20 @@ Linux example: ```bash ./build.sh --cmake_generator "Ninja" --android --android_sdk_path /Android --android_ndk_path /Android/ndk/21.1.6352462/ --android_abi arm64-v8a --android_api 29 --use_xnnpack ``` +### Build for iOS (available since 1.14) +A Mac machine is required to build package for iOS. Please follow this [guide](./ios.md) to set up environment firstly. +#### Create a minimal build with XNNPACK EP support + +Once you have all the necessary components setup, follow the instructions to [create the custom build](./custom.md), with the following changes: + +* Replace `--minimal_build` with `--minimal_build extended` to enable support for execution providers that dynamically create kernels at runtime, which is required by the XNNPACK EP. +* Add `--use_xnnpack` to include the XNNPACK EP in the build + +```dos +./build.sh --config --use_xcode \ + --ios --ios_sysroot iphoneos --osx_arch arm64 --apple_deploy_target --use_xnnpack --minimal_build extended --disable_ml_ops --disable_exceptions --build_shared_lib --skip_tests --include_ops_by_config +``` + ### Build for Windows ```dos .\build.bat --config --use_xnnpack diff --git a/docs/execution-providers/Xnnpack-ExecutionProvider.md b/docs/execution-providers/Xnnpack-ExecutionProvider.md index 445e9a2d99..82d2d8524c 100644 --- a/docs/execution-providers/Xnnpack-ExecutionProvider.md +++ b/docs/execution-providers/Xnnpack-ExecutionProvider.md @@ -8,7 +8,7 @@ nav_order: 9 # XNNPACK Execution Provider -Accelerate ONNX models on Android devices and WebAssembly with ONNX Runtime and the XNNPACK execution provider. [(XNNPACK)](https://github.com/google/XNNPACK) is a highly optimized library of floating-point neural network inference operators for ARM, WebAssembly, and x86 platforms. +Accelerate ONNX models on Android/iOS devices and WebAssembly with ONNX Runtime and the XNNPACK execution provider. [(XNNPACK)](https://github.com/google/XNNPACK) is a highly optimized library of floating-point neural network inference operators for ARM, WebAssembly, and x86 platforms. ## Contents {: .no_toc } @@ -23,12 +23,14 @@ Accelerate ONNX models on Android devices and WebAssembly with ONNX Runtime and Pre-built packages of ONNX Runtime([`onnxruntime-android`](https://mvnrepository.com/artifact/com.microsoft.onnxruntime/onnxruntime-android)) with XNNPACK EP for Android are published on Maven. See [here](../install/index.md#install-on-android) for installation instructions. -There is no pre-built package for iOS yet. We will update it once it is ready. +Pre-built binaries(`onnxruntime-objc` and `onnxruntime-c`) of ONNX Runtime with XNNPACK EP for iOS are published to CocoaPods. +See [here](../install/index.md#install-on-ios) for installation instructions. + ## Build Please see the [Build page](../build/eps.md#xnnpack) for instructions on building a package that includes the XNNPACK EP. -Support build for Android/Windows/Linux +Support build for Android/iOS/Windows/Linux ## Usage @@ -75,12 +77,17 @@ The number of threads to use for the XNNPACK EP's internal intra-op thread-pool. ## Supported ops Following ops are supported by the XNNPACK Execution Provider, -|Operator|Note| -|--------|------| +|Operator|Note|| +|--------|------|-----| |ai.onnx:AveragePool|Only 2D Pool is supported.| |ai.onnx:Conv|Only 2D Conv is supported.
Weights and bias should be constant.| +|ai.onnx:ConvTranspose|Only 2D ConvTranspose is supported.
Weights and bias should be constant.|since 1.14| |ai.onnx:MaxPool|Only 2D Pool is supported.| |ai.onnx:Softmax|all opset below 13 is supported, only support opset 13 when AXIS is the last dimension| |ai.onnx:QLinearConv|Only 2D Conv is supported.
Weights and bias should be constant.
All quantization scales and zero points should be constant.| +|ai.onnx:Resize|2D/4D Resize in `Bilinear mode` are supported|since 1.14| +|ai.onnx:Gemm|Only 2D Op is supported|since 1.14| +|ai.onnx:Matmul|Only 2D Op is supported|since 1.14| |com.microsoft:QLinearAveragePool|Only 2D Pool is supported.
All quantization scales and zero points should be constant.| |com.microsoft:QLinearSoftmax|All quantization scales and zero points should be constant.| +|com.microsoft:QLinearConvTranspose|All quantization scales and zero points should be constant.|since 1.14| diff --git a/js/script.js b/js/script.js index 3e1a932302..d81ffcd28a 100644 --- a/js/script.js +++ b/js/script.js @@ -1169,6 +1169,12 @@ var validCombos = { "Follow build instructions from here", //Xnnpack + "ios,C-API,ARM64,XNNPACK": + "Add 'onnxruntime-c' using CocoaPods and refer to the mobile deployment guide or Follow build instructions from here", + + "ios,objectivec,ARM64,XNNPACK": + "Add 'onnxruntime-objc' using CocoaPods and refer to the mobile deployment guide", + "android,C-API,ARM64,XNNPACK": "Follow build instructions from here",