diff --git a/docs/api/csharp-api.md b/docs/api/csharp-api.md index 52341b08f9..9717c2f540 100644 --- a/docs/api/csharp-api.md +++ b/docs/api/csharp-api.md @@ -8,6 +8,8 @@ redirect_from: /docs/reference/api/csharp-api # C# API Reference {: .no_toc } +This page shows the main elements of the C# API for ONNX Runtime. + ### OrtEnv ```cs class OrtEnv diff --git a/docs/build/android-ios.md b/docs/build/android.md similarity index 65% rename from docs/build/android-ios.md rename to docs/build/android.md index 60cf990c8b..137d797153 100644 --- a/docs/build/android-ios.md +++ b/docs/build/android.md @@ -1,13 +1,15 @@ --- -title: Build for Android/iOS -parent: Build ORT +title: Build for Android +parent: Build ONNX Runtime nav_order: 5 +redirect_from: /docs/how-to/build/android-ios,/docs/build/android-ios --- -` -# Build ONNX Runtime for Android and iOS + +# Build ONNX Runtime for Android {: .no_toc } -Below are general build instructions for Android and iOS. For examples of deploying ONNX Runtime on mobile platforms (includes overall smaller package size and other configurations), see [Mobile Tutorials](../tutorials/mobile). +Follow the instructions below to build ONNX Runtime for Android. + ## Contents {: .no_toc } @@ -15,9 +17,7 @@ Below are general build instructions for Android and iOS. For examples of deploy * TOC placeholder {:toc} -## Android - -### Prerequisites +## Prerequisites The SDK and NDK packages can be installed via Android Studio or the sdkmanager command line tool. @@ -29,7 +29,7 @@ Resources: - [Android ABIs](https://developer.android.com/ndk/guides/abis) - [System Images](https://developer.android.com/topic/generic-system-image) -#### Android Studio +### Android Studio 1. [Install](https://developer.android.com/studio) Android Studio @@ -45,7 +45,7 @@ Resources: * The NDK path will be the 'ndk/{version}' subdirectory of the SDK path shown * e.g. if 21.1.6352462 is installed it will be {SDK path}/ndk/21.1.6352462 -#### sdkmanager from command line tools +### sdkmanager from command line tools * If necessary install the Java Runtime Environment and set the JAVA_HOME environment variable to point to it * https://www.java.com/en/download/ @@ -83,9 +83,9 @@ Resources: * NDK path in our example with this install would be `.../Android/ndk/21.1.6352462` * NOTE: If you install the ndk-bundle package the path will be `.../Android/ndk-bundle` as there's no version number -### Android Build Instructions +## Android Build Instructions -#### Cross compiling on Windows +### Cross compiling on Windows The [Ninja](https://ninja-build.org/) generator needs to be used to build on Windows as the Visual Studio generator doesn't support Android. @@ -99,13 +99,13 @@ e.g. using the paths from our example ./build.bat --android --android_sdk_path .../Android --android_ndk_path .../Android/ndk/21.1.6352462 --android_abi arm64-v8a --android_api 27 --cmake_generator Ninja ``` -#### Cross compiling on Linux and macOS +### Cross compiling on Linux and macOS ```bash ./build.sh --android --android_sdk_path --android_ndk_path --android_abi --android_api ``` -#### Build Android Archive (AAR) +### Build Android Archive (AAR) Android Archive (AAR) files, which can be imported directly in Android Studio, will be generated in your_build_dir/java/build/android/outputs/aar, by using the above building commands with `--build_java` @@ -119,86 +119,11 @@ To build on Windows with `--build_java` enabled you must also: * run the build from an admin window * the Java build needs permissions to create a symlink, which requires an admin window -### Android NNAPI Execution Provider +## Android NNAPI Execution Provider If you want to use NNAPI Execution Provider on Android, see [NNAPI Execution Provider](../execution-providers/NNAPI-ExecutionProvider). -#### Build Instructions +### Build Instructions Android NNAPI Execution Provider can be built using building commands in [Android Build instructions](#android-build-instructions) with `--use_nnapi` ---- - -## iOS - -### Prerequisites - -* A Mac computer with latest macOS -* Xcode, https://developer.apple.com/xcode/ -* CMake, https://cmake.org/download/ -* Python 3, https://www.python.org/downloads/mac-osx/ - -### General Info - -* iOS Platforms - - The following two platforms are supported - * iOS device (iPhone, iPad) with arm64 architecture - * iOS simulator with x86_64 architecture - - The following platforms are *not* supported - * armv7 - * armv7s - * i386 architectures - * tvOS - * watchOS platforms are not currently supported. - -* apple_deploy_target - - Specify the minimum version of the target platform (iOS) on which the target binaries are to be deployed. - -* Code Signing - - If the code signing development team ID or code signing identity is specified, and has a valid code signing certificate, Xcode will code sign the onnxruntime library in the building process. Otherwise, the onnxruntime will be built without code signing. It may be required or desired to code sign the library for iOS devices. For more information, see [Code Signing](https://developer.apple.com/support/code-signing/). - -### Build Instructions - -Run one of the following build scripts from the ONNX Runtime repository root: - -#### Cross build for iOS simulator - -```bash -./build.sh --config --use_xcode \ - --ios --ios_sysroot iphonesimulator --osx_arch x86_64 --apple_deploy_target -``` - -#### Cross build for iOS device - -```bash -./build.sh --config --use_xcode \ - --ios --ios_sysroot iphoneos --osx_arch arm64 --apple_deploy_target -``` - -#### Cross build for iOS device and code sign the library using development team ID - -```bash -./build.sh --config --use_xcode \ - --ios --ios_sysroot iphoneos --osx_arch arm64 --apple_deploy_target \ - --xcode_code_signing_team_id -``` - -#### Cross build for iOS device and code sign the library using code sign identity - -```bash -./build.sh --config --use_xcode \ - --ios --ios_sysroot iphoneos --osx_arch arm64 --apple_deploy_target \ - --xcode_code_signing_identity -``` - -### CoreML Execution Provider - -If you want to use CoreML Execution Provider on iOS or macOS, see [CoreML Execution Provider](../execution-providers/CoreML-ExecutionProvider). - -#### Build Instructions - -CoreML Execution Provider can be built using building commands in [iOS Build instructions](#build-instructions-1) with `--use_coreml` diff --git a/docs/build/eps.md b/docs/build/eps.md index f0764caa9a..9a9c16635a 100644 --- a/docs/build/eps.md +++ b/docs/build/eps.md @@ -1,7 +1,9 @@ --- title: Build with different EPs -parent: Build ORT +parent: Build ONNX Runtime +description: Learm how to build ONNX Runtime from source for different execution providers nav_order: 3 +redirect_from: /docs/how-to/build/eps --- # Build ONNX Runtime with Execution Providers diff --git a/docs/build/index.md b/docs/build/index.md index 9adfcd48f5..2f2d358712 100644 --- a/docs/build/index.md +++ b/docs/build/index.md @@ -1,8 +1,11 @@ --- -title: Build ORT +title: Build ONNX Runtime has_children: true nav_order: 5 +redirect_from: /docs/how-to/build --- + # Build ONNX Runtime from source {: .no_toc } + For production deployments, it's strongly recommended to build only from an [official release branch](https://github.com/microsoft/onnxruntime/releases). diff --git a/docs/build/inferencing.md b/docs/build/inferencing.md index 8d57f1d118..a7e3296412 100644 --- a/docs/build/inferencing.md +++ b/docs/build/inferencing.md @@ -1,12 +1,15 @@ --- title: Build for inferencing -parent: Build ORT +parent: Build ONNX Runtime nav_order: 1 +redirect_from: /docs/how-to/build/inferencing --- # Build ONNX Runtime for inferencing {: .no_toc } +Follow the instructions below to build ONNX Runtime to perform inference. + ## Contents {: .no_toc } @@ -512,8 +515,8 @@ ls -l /code/onnxruntime/build/Linux/MinSizeRel/dist/*.whl --- -### Android/iOS -Please see [Build for Android/iOS](./android-ios.md) +### Mobile +Please see [Build for Android](./android.md) and [Build for iOS](./ios.md) ### Web Please see [Build for Web](./web.md) diff --git a/docs/build/ios.md b/docs/build/ios.md new file mode 100644 index 0000000000..d59da628de --- /dev/null +++ b/docs/build/ios.md @@ -0,0 +1,90 @@ +--- +title: Build for iOS +parent: Build ONNX Runtime +nav_order: 5 +--- + +# Build ONNX Runtime for iOS +{: .no_toc } + +Follow the instructions below to build ONNX Runtime for iOS. + + +## Contents +{: .no_toc } + +* TOC placeholder +{:toc} + +## General Info + +* iOS Platforms + + The following two platforms are supported + * iOS device (iPhone, iPad) with arm64 architecture + * iOS simulator with x86_64 architecture + + The following platforms are *not* supported + * armv7 + * armv7s + * i386 architectures + * tvOS + * watchOS platforms are not currently supported. + +* apple_deploy_target + + Specify the minimum version of the target platform (iOS) on which the target binaries are to be deployed. + +* Code Signing + + If the code signing development team ID or code signing identity is specified, and has a valid code signing certificate, Xcode will code sign the onnxruntime library in the building process. Otherwise, the onnxruntime will be built without code signing. It may be required or desired to code sign the library for iOS devices. For more information, see [Code Signing](https://developer.apple.com/support/code-signing/). + + +## Prerequisites + +* A Mac computer with latest macOS +* Xcode, https://developer.apple.com/xcode/ +* CMake, https://cmake.org/download/ +* Python 3, https://www.python.org/downloads/mac-osx/ + +## Build Instructions + +Run one of the following build scripts from the ONNX Runtime repository root: + +### Cross compile for iOS simulator + +```bash +./build.sh --config --use_xcode \ + --ios --ios_sysroot iphonesimulator --osx_arch x86_64 --apple_deploy_target +``` + +### Cross compile for iOS device + +```bash +./build.sh --config --use_xcode \ + --ios --ios_sysroot iphoneos --osx_arch arm64 --apple_deploy_target +``` + +### Cross compile for iOS device and code sign the library using development team ID + +```bash +./build.sh --config --use_xcode \ + --ios --ios_sysroot iphoneos --osx_arch arm64 --apple_deploy_target \ + --xcode_code_signing_team_id +``` + +### Cross compile for iOS device and code sign the library using code sign identity + +```bash +./build.sh --config --use_xcode \ + --ios --ios_sysroot iphoneos --osx_arch arm64 --apple_deploy_target \ + --xcode_code_signing_identity +``` + +### CoreML Execution Provider + +If you want to use CoreML Execution Provider on iOS or macOS, see [CoreML Execution Provider](../execution-providers/CoreML-ExecutionProvider). + +#### Build Instructions + +CoreML Execution Provider can be built using building commands in [iOS Build instructions](#build-instructions-1) with `--use_coreml` \ No newline at end of file diff --git a/docs/build/reduced.md b/docs/build/reduced.md index 0c7ad9c5c6..ac633fb2c2 100644 --- a/docs/build/reduced.md +++ b/docs/build/reduced.md @@ -1,15 +1,16 @@ --- title: Build with reduced size -parent: Build ORT +parent: Build ONNX Runtime nav_order: 6 +redirect_from: /docs/how-to/build/reduced --- -# Build ORT with reduced size +# Reduce the size of the ONNX Runtime build {: .no_toc } For applications where package binary size is important, ONNX Runtime provides options to reduce the build size with some functional trade-offs. -To reduce the compiled binary size of ONNX Runtime, the operator kernels included in the build can be reduced to just the kernels required by your model/s. +To reduce the compiled binary size of ONNX Runtime, the build can be configured to only include the operator kernels needed by your model(s). For deployment on mobile devices specifically, please read more detailed guidance on [Deploy ONNX Runtime Mobile](../tutorials/mobile/). diff --git a/docs/build/training.md b/docs/build/training.md index 049a5ebd09..ca4dd1ed2d 100644 --- a/docs/build/training.md +++ b/docs/build/training.md @@ -1,7 +1,9 @@ --- title: Build for training -parent: Build ORT +parent: Build ONNX Runtime +description: Learn how to build ONNX Runtime for training from source for different hardware targets nav_order: 2 +redirect_from: /docs/how-to/build/training --- # Build ONNX Runtime for training diff --git a/docs/build/web.md b/docs/build/web.md index f9b0bec214..75086ab405 100644 --- a/docs/build/web.md +++ b/docs/build/web.md @@ -1,7 +1,9 @@ --- title: Build for web -parent: Build ORT +parent: Build ONNX Runtime +description: Learn how to build ONNX Runtime from source to deploy on the web nav_order: 4 +redirect_from: /docs/how-to/build/web --- # Build ONNX Runtime for Web diff --git a/docs/ecosystem/index.md b/docs/ecosystem/index.md index 9d3da18692..9e0430d93d 100644 --- a/docs/ecosystem/index.md +++ b/docs/ecosystem/index.md @@ -1,6 +1,8 @@ --- -title: ORT Ecosystem +title: ONNX Runtime Ecosystem +description: See examples of how ONNX Runtime working end to end within the Azure AI and ML landscape and ecosystem nav_order: 11 +redirect_from: /docs/tutorials/ecosystem --- # ORT Ecosystem {: .no_toc } diff --git a/docs/execution-providers/ACL-ExecutionProvider.md b/docs/execution-providers/ACL-ExecutionProvider.md index aafe30f8dd..e0b6361700 100644 --- a/docs/execution-providers/ACL-ExecutionProvider.md +++ b/docs/execution-providers/ACL-ExecutionProvider.md @@ -1,13 +1,15 @@ --- -title: ARM Compute Library +title: ARM Compute Library (ACL) +description: Instructions to execute ONNX Runtime with the ACL Execution Provider parent: Execution Providers nav_order: 4 +redirect_from: /docs/reference/execution-providers/ACL-ExecutionProvider --- # ACL Execution Provider {: .no_toc } -[Arm Compute Library](https://github.com/ARM-software/ComputeLibrary){:target="_blank"} is an open source inference engine maintained by Arm and Linaro companies. The integration of ACL as an execution provider (EP) into ONNX Runtime accelerates performance of ONNX model workloads across Armv8 cores. +The integration of ACL as an execution provider (EP) into ONNX Runtime accelerates performance of ONNX model workloads across Armv8 cores. [Arm Compute Library](https://github.com/ARM-software/ComputeLibrary){:target="_blank"} is an open source inference engine maintained by Arm and Linaro companies. ## Contents diff --git a/docs/execution-providers/ArmNN-ExecutionProvider.md b/docs/execution-providers/ArmNN-ExecutionProvider.md index 9c234215a7..a7828a68a4 100644 --- a/docs/execution-providers/ArmNN-ExecutionProvider.md +++ b/docs/execution-providers/ArmNN-ExecutionProvider.md @@ -1,7 +1,9 @@ --- -title: ARM NN +title: Arm NN +description: Instructions to execute ONNX Runtime with Arm NN on Armv8 cores parent: Execution Providers nav_order: 3 +redirect_from: /docs/reference/execution-providers/ArmNN-ExecutionProvider --- # ArmNN Execution Provider @@ -13,7 +15,7 @@ nav_order: 3 * TOC placeholder {:toc} -[ArmNN](https://github.com/ARM-software/armnn) is an open source inference engine maintained by Arm and Linaro companies. The integration of ArmNN as an execution provider (EP) into ONNX Runtime accelerates performance of ONNX model workloads across Armv8 cores. +Accelerate performance of ONNX model workloads across Armv8 cores with the ArmNN execution provider. [ArmNN](https://github.com/ARM-software/armnn) is an open source inference engine maintained by Arm and Linaro companies. ## Build For build instructions, please see the [BUILD page](../build/eps.md#armnn). diff --git a/docs/execution-providers/CUDA-ExecutionProvider.md b/docs/execution-providers/CUDA-ExecutionProvider.md index 8658dce1c1..359d3b5045 100644 --- a/docs/execution-providers/CUDA-ExecutionProvider.md +++ b/docs/execution-providers/CUDA-ExecutionProvider.md @@ -1,7 +1,9 @@ --- title: CUDA +description: Instructions to execute ONNX Runtime applications with CUDA parent: Execution Providers nav_order: 1 +redirect_from: /docs/reference/execution-providers/CUDA-ExecutionProvider --- # CUDA Execution Provider diff --git a/docs/execution-providers/CoreML-ExecutionProvider.md b/docs/execution-providers/CoreML-ExecutionProvider.md index 3c24f666f1..43293e5da6 100644 --- a/docs/execution-providers/CoreML-ExecutionProvider.md +++ b/docs/execution-providers/CoreML-ExecutionProvider.md @@ -1,7 +1,9 @@ --- title: CoreML +description: Instructions to execute ONNX Runtime with CoreML parent: Execution Providers nav_order: 2 +redirect_from: /docs/reference/execution-providers/CoreML-ExecutionProvider --- {::options toc_levels="2" /} @@ -29,7 +31,7 @@ Pre-built binaries of ONNX Runtime Mobile with CoreML EP for iOS are published t ## Build -For build instructions for iOS devices, please see [How to: Build for Android/iOS](../build/android-ios.md#coreml-execution-provider). +For build instructions for iOS devices, please see [Build for iOS](../build/ios.md#coreml-execution-provider). ## Usage diff --git a/docs/execution-providers/DirectML-ExecutionProvider.md b/docs/execution-providers/DirectML-ExecutionProvider.md index 2dec00b60f..fc2428eea1 100644 --- a/docs/execution-providers/DirectML-ExecutionProvider.md +++ b/docs/execution-providers/DirectML-ExecutionProvider.md @@ -1,18 +1,20 @@ --- title: Direct ML +description: Instructions to execute ONNX Runtime with the DirectML execution provider parent: Execution Providers nav_order: 5 +redirect_from: /docs/reference/execution-providers/DirectML-ExecutionProvider --- # DirectML Execution Provider {: .no_toc } +The DirectML Execution Provider is a component of ONNX Runtime that uses DirectML to accelerate inference of ONNX models. The DirectML execution provider is capable of greatly improving evaluation time of models using commodity GPU hardware, without sacrificing broad hardware support or requiring vendor-specific extensions to be installed. + DirectML is a high-performance, hardware-accelerated DirectX 12 library for machine learning on Windows. DirectML provides GPU acceleration for common machine learning tasks across a broad range of supported hardware and drivers. When used standalone, the DirectML API is a low-level DirectX 12 library and is suitable for high-performance, low-latency applications such as frameworks, games, and other real-time applications. The seamless interoperability of DirectML with Direct3D 12 as well as its low overhead and conformance across hardware makes DirectML ideal for accelerating machine learning when both high performance is desired, and the reliability and predictability of results across hardware is critical. -The *DirectML Execution Provider* is an optional component of ONNX Runtime that uses DirectML to accelerate inference of ONNX models. The DirectML execution provider is capable of greatly improving evaluation time of models using commodity GPU hardware, without sacrificing broad hardware support or requiring vendor-specific extensions to be installed. - The DirectML Execution Provider currently uses DirectML version 1.4.2. ## Contents diff --git a/docs/execution-providers/MIGraphX-ExecutionProvider.md b/docs/execution-providers/MIGraphX-ExecutionProvider.md index 311840bdd5..eec3cceffd 100644 --- a/docs/execution-providers/MIGraphX-ExecutionProvider.md +++ b/docs/execution-providers/MIGraphX-ExecutionProvider.md @@ -1,7 +1,9 @@ --- -title: AMD MI GraphX +title: AMD MIGraphX +description: Instructions to execute ONNX Runtime with the AMD MIGraphX execution provider parent: Execution Providers nav_order: 6 +redirect_from: /docs/reference/execution-providers/MIGraphX-ExecutionProvider --- # MIGraphX Execution Provider diff --git a/docs/execution-providers/NNAPI-ExecutionProvider.md b/docs/execution-providers/NNAPI-ExecutionProvider.md index 6a307c75c1..93ef57a9a9 100644 --- a/docs/execution-providers/NNAPI-ExecutionProvider.md +++ b/docs/execution-providers/NNAPI-ExecutionProvider.md @@ -1,13 +1,15 @@ --- title: NNAPI +description: Instructions to execute ONNX Runtime with the NNAPI execution provider parent: Execution Providers nav_order: 7 +redirect_from: /docs/reference/execution-providers/NNAPI-ExecutionProvider --- {::options toc_levels="2" /} # NNAPI Execution Provider -[Android Neural Networks API (NNAPI)](https://developer.android.com/ndk/guides/neuralnetworks) is a unified interface to CPU, GPU, and NN accelerators on Android. +Accelerate ONNX models on Android devices with ONNX Runtime and the NNAPI execution provider. [Android Neural Networks API (NNAPI)](https://developer.android.com/ndk/guides/neuralnetworks) is a unified interface to CPU, GPU, and NN accelerators on Android. ## Contents {: .no_toc } diff --git a/docs/execution-providers/Nuphar-ExecutionProvider.md b/docs/execution-providers/Nuphar-ExecutionProvider.md index 935afaabff..c4daef91b9 100644 --- a/docs/execution-providers/Nuphar-ExecutionProvider.md +++ b/docs/execution-providers/Nuphar-ExecutionProvider.md @@ -1,15 +1,19 @@ --- title: NUPHAR +description: Instructions to execute ONNX Runtime with the Nuphar execution provider parent: Execution Providers nav_order: 8 +redirect_from: /docs/reference/execution-providers/Nuphar-ExecutionProvider --- # Nuphar Execution Provider {: .no_toc } +Tap into the power of Nuphar through ONNX Runtime to accelerate inferencing of ONNX models. + NUPHAR stands for Neural-network Unified Preprocessing Heterogeneous Architecture. As an execution provider in the ONNX Runtime, it is built on top of [TVM](https://github.com/dmlc/tvm) and [LLVM](https://llvm.org) to accelerate ONNX models by compiling nodes in subgraphs into optimized functions via JIT. It also provides JIT caching to save compilation time at runtime. -Developers can tap into the power of Nuphar through ONNX Runtime to accelerate inferencing of ONNX models. The Nuphar execution provider comes with a common ONNX to TVM lowering [library](https://github.com/microsoft/onnxruntime/tree/master/onnxruntime/core/codegen) that can potentially be reused by other execution providers to leverage TVM. With the Nuphar execution provider, the ONNX Runtime delivers better inferencing performance on the same hardware compared to generic X64 CPU acceleration, especially for quantized recurrent neural networks. Various products at Microsoft have seen up to a 5x improvement in performance with no loss of accuracy, by running quantized LSTMs via the Nuphar execution provider in the ONNX Runtime. +The Nuphar execution provider comes with a common ONNX to TVM lowering [library](https://github.com/microsoft/onnxruntime/tree/master/onnxruntime/core/codegen) that can potentially be reused by other execution providers to leverage TVM. With the Nuphar execution provider, the ONNX Runtime delivers better inferencing performance on the same hardware compared to generic X64 CPU acceleration, especially for quantized recurrent neural networks. Various products at Microsoft have seen up to a 5x improvement in performance with no loss of accuracy, by running quantized LSTMs via the Nuphar execution provider in the ONNX Runtime. ## Contents {: .no_toc } diff --git a/docs/execution-providers/OpenVINO-ExecutionProvider.md b/docs/execution-providers/OpenVINO-ExecutionProvider.md index 3cf9bfdf65..14416c9b60 100644 --- a/docs/execution-providers/OpenVINO-ExecutionProvider.md +++ b/docs/execution-providers/OpenVINO-ExecutionProvider.md @@ -1,13 +1,15 @@ --- title: OpenVINO +description: Instructions to execute ONNX Runtime with the Intel OpenVINO execution provider parent: Execution Providers nav_order: 10 +redirect_from: /docs/reference/execution-providers/OpenVINO-ExecutionProvider --- # OpenVINO Execution Provider {: .no_toc } -OpenVINO Execution Provider enables deep learning inference on Intel CPUs, Intel integrated GPUs and Intel® MovidiusTM Vision Processing Units (VPUs). Please refer to [this](https://software.intel.com/en-us/openvino-toolkit/hardware) page for details on the Intel hardware supported. +Accelerate ONNX models on Intel CPUs, GPUs and VPUs with ONNX Runtime and the Intel OpenVINO execution provider. Please refer to [this](https://software.intel.com/en-us/openvino-toolkit/hardware) page for details on the Intel hardware supported. ## Contents {: .no_toc } diff --git a/docs/execution-providers/RKNPU-ExecutionProvider.md b/docs/execution-providers/RKNPU-ExecutionProvider.md index e6b573bbd6..a14dee74a4 100644 --- a/docs/execution-providers/RKNPU-ExecutionProvider.md +++ b/docs/execution-providers/RKNPU-ExecutionProvider.md @@ -1,7 +1,9 @@ --- title: RKNPU +description: Instructions to execute ONNX Runtime on Rockchip NPUs with the RKNPU execution provider parent: Execution Providers nav_order: 11 +redirect_from: /docs/reference/execution-providers/RKNPU-ExecutionProvider --- # RKNPU Execution Provider diff --git a/docs/execution-providers/TensorRT-ExecutionProvider.md b/docs/execution-providers/TensorRT-ExecutionProvider.md index 66982d999e..ecc8a6c206 100644 --- a/docs/execution-providers/TensorRT-ExecutionProvider.md +++ b/docs/execution-providers/TensorRT-ExecutionProvider.md @@ -1,16 +1,18 @@ --- title: TensorRT +description: Instructions to execute ONNX Runtime on NVIDIA GPUs with the TensorRT execution provider parent: Execution Providers nav_order: 12 +redirect_from: /docs/reference/execution-providers/TensorRT-ExecutionProvider --- # TensorRT Execution Provider {: .no_toc } -The TensorRT execution provider in the ONNX Runtime makes use of NVIDIA's [TensorRT](https://developer.nvidia.com/tensorrt) Deep Learning inferencing engine to accelerate ONNX model in their family of GPUs. Microsoft and NVIDIA worked closely to integrate the TensorRT execution provider with ONNX Runtime. - With the TensorRT execution provider, the ONNX Runtime delivers better inferencing performance on the same hardware compared to generic GPU acceleration. +The TensorRT execution provider in the ONNX Runtime makes use of NVIDIA's [TensorRT](https://developer.nvidia.com/tensorrt) Deep Learning inferencing engine to accelerate ONNX model in their family of GPUs. Microsoft and NVIDIA worked closely to integrate the TensorRT execution provider with ONNX Runtime. + ## Contents {: .no_toc } diff --git a/docs/execution-providers/Vitis-AI-ExecutionProvider.md b/docs/execution-providers/Vitis-AI-ExecutionProvider.md index 97695678ea..08ce51931a 100644 --- a/docs/execution-providers/Vitis-AI-ExecutionProvider.md +++ b/docs/execution-providers/Vitis-AI-ExecutionProvider.md @@ -1,7 +1,9 @@ --- title: Vitis AI +description: Instructions to execute ONNX Runtime on Xilinx devices with the Vitis AI execution provider parent: Execution Providers nav_order: 13 +redirect_from: /docs/reference/execution-providers/Vitis-AI-ExecutionProvider --- # Vitis-AI Execution Provider diff --git a/docs/execution-providers/add-execution-provider.md b/docs/execution-providers/add-execution-provider.md index 70e1065b85..486118278a 100644 --- a/docs/execution-providers/add-execution-provider.md +++ b/docs/execution-providers/add-execution-provider.md @@ -1,7 +1,9 @@ --- title: Add a new execution provider +description: Instructions to add a new execution provider to ONNX Runtime parent: Execution Providers nav_order: 14 +redirect_from: /docs/how-to/add-execution-provider --- # Add a new Execution Provider to ONNX Runtime diff --git a/docs/execution-providers/oneDNN-ExecutionProvider.md b/docs/execution-providers/oneDNN-ExecutionProvider.md index 42a89e9934..2b7ba927c6 100644 --- a/docs/execution-providers/oneDNN-ExecutionProvider.md +++ b/docs/execution-providers/oneDNN-ExecutionProvider.md @@ -1,17 +1,21 @@ --- title: Intel oneDNN +description: Instructions to execute ONNX Runtime with the Intel oneDNN execution provider parent: Execution Providers nav_order: 9 +redirect_from: /docs/reference/execution-providers/oneDNN-ExecutionProvider --- # oneDNN Execution Provider {: .no_toc } -*Formerly "DNNL"* +Accelerate performance of ONNX Runtime using Intel® Math Kernel Library for Deep Neural Networks (Intel® DNNL) optimized primitives with the Intel oneDNN execution provider. Intel® oneAPI Deep Neural Network Library is an open-source performance library for deep-learning applications. The library accelerates deep-learning applications and frameworks on Intel® architecture and Intel® Processor Graphics Architecture. Intel DNNL contains vectorized and threaded building blocks that you can use to implement deep neural networks (DNN) with C and C++ interfaces. -Intel and Microsoft have developed the DNNL Execution Provider (EP) for ONNX Runtime to accelerate performance of ONNX Runtime using Intel® Math Kernel Library for Deep Neural Networks (Intel® DNNL) optimized primitives. +The DNNL Execution Provider (EP) for ONNX Runtime is developed by a partnership between Intel and Microsoft. + +*Formerly "DNNL"* ## Contents {: .no_toc } diff --git a/docs/index.md b/docs/index.md index 772188bee0..986f07ff19 100644 --- a/docs/index.md +++ b/docs/index.md @@ -19,7 +19,7 @@ ONNX Runtime is an accelerator for machine learning models with multi platform s ## Contribute and Customize |:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------| -| [Build ORT Packages](./build){: .btn .mr-4 target="_blank"} | [ORT GitHub](https://github.com/microsoft/onnxruntime){: .btn target="_blank"} | +| [Build ORT Packages](./build){: .btn .mr-4 target="_blank"} | ONNX Runtime GitHub](https://github.com/microsoft/onnxruntime){: .btn target="_blank"} | --- diff --git a/docs/install/index.md b/docs/install/index.md index ac8ea67ebd..bd02550cd9 100644 --- a/docs/install/index.md +++ b/docs/install/index.md @@ -1,7 +1,9 @@ --- -title: Install ORT +title: Install ONNX Runtime +description: Instructions to install ONNX Runtime on your target platform in your environment has_children: false nav_order: 1 +redirect_from: /docs/how-to/install --- # Install ONNX Runtime (ORT) diff --git a/docs/performance/graph-optimizations.md b/docs/performance/graph-optimizations.md index 86b1a02067..0e04fa415a 100644 --- a/docs/performance/graph-optimizations.md +++ b/docs/performance/graph-optimizations.md @@ -1,7 +1,8 @@ --- title: Graph optimizations parent: Performance -nav_order: 2 +nav_order: 3 +redirect_from: /docs/how-to/graph-optimizations --- # Graph Optimizations in ONNX Runtime diff --git a/docs/performance/mobile-performance-tuning.md b/docs/performance/mobile-performance-tuning.md index 5bee1119ff..06311855d8 100644 --- a/docs/performance/mobile-performance-tuning.md +++ b/docs/performance/mobile-performance-tuning.md @@ -1,13 +1,16 @@ --- -title: ONNX Runtime Mobile Performance Tuning +title: Tune Mobile Performance parent: Performance -nav_order: 1 +nav_order: 2 +redirect_from: /docs/how-to/mobile-performance-tuning --- {::options toc_levels="2" /} -# Performance Tuning +# ONNX Runtime Mobile Performance Tuning -ONNX Runtime Mobile can be used to execute ORT format models using NNAPI (via the NNAPI Execution Provider (EP)) on Android platforms, and CoreML (via the CoreML EP) on iOS platforms. This section explains the details of how different optimizations affect performance, and provides some suggestions for performance testing with ORT format models. +Learn how different optimizations affect performance, and get suggestions for performance testing with ORT format models. + +ONNX Runtime Mobile can be used to execute ORT format models using NNAPI (via the NNAPI Execution Provider (EP)) on Android platforms, and CoreML (via the CoreML EP) on iOS platforms. First, please review the introductory details in [using NNAPI and CoreML with ONNX Runtime Mobile](../tutorials/mobile/using-nnapi-coreml-with-ort-mobile.md). diff --git a/docs/performance/quantization.md b/docs/performance/quantization.md index d065381965..09df778e15 100644 --- a/docs/performance/quantization.md +++ b/docs/performance/quantization.md @@ -2,6 +2,7 @@ title: Quantize ONNX Models parent: Performance nav_order: 4 +redirect_from: /docs/how-to/quantization --- # Quantize ONNX Models {: .no_toc } diff --git a/docs/performance/tune-performance.md b/docs/performance/tune-performance.md index 20bcbf7657..58c1b90651 100644 --- a/docs/performance/tune-performance.md +++ b/docs/performance/tune-performance.md @@ -1,7 +1,8 @@ --- title: Tune performance parent: Performance -nav_order: 3 +nav_order: 1 +redirect_from: /docs/how-to/tune-performance --- # ONNX Runtime Performance Tuning diff --git a/docs/tutorials/api-basics.md b/docs/tutorials/api-basics.md index 44548750f1..a257e68cf7 100644 --- a/docs/tutorials/api-basics.md +++ b/docs/tutorials/api-basics.md @@ -1,5 +1,6 @@ --- title: API Basics +descriptions: parent: Tutorials nav_order: 1 --- diff --git a/docs/tutorials/mobile/limitations.md b/docs/tutorials/mobile/limitations.md index 79aec3d242..5d19ecdac5 100644 --- a/docs/tutorials/mobile/limitations.md +++ b/docs/tutorials/mobile/limitations.md @@ -5,7 +5,7 @@ grand_parent: Tutorials nav_order: 7 --- -## Limitations +# Limitations A minimal build has the following limitations: - No support for ONNX format models diff --git a/docs/tutorials/mobile/model-conversion.md b/docs/tutorials/mobile/model-conversion.md index a0cccf835d..1e07242caf 100644 --- a/docs/tutorials/mobile/model-conversion.md +++ b/docs/tutorials/mobile/model-conversion.md @@ -8,7 +8,7 @@ nav_order: 3 * TOC {:toc} -## Converting ONNX models to ORT format +# Converting ONNX models to ORT format Conversion of ONNX format models to ORT format utilizes the ONNX Runtime python package, as the model is loaded into ONNX Runtime and optimized as part of the conversion process. diff --git a/docs/tutorials/mobile/using-nnapi-coreml-with-ort-mobile.md b/docs/tutorials/mobile/using-nnapi-coreml-with-ort-mobile.md index 9cd1d3d3d9..6ab8ebad58 100644 --- a/docs/tutorials/mobile/using-nnapi-coreml-with-ort-mobile.md +++ b/docs/tutorials/mobile/using-nnapi-coreml-with-ort-mobile.md @@ -5,7 +5,7 @@ grand_parent: Tutorials nav_order: 6 --- -## Using NNAPI and CoreML with ONNX Runtime Mobile +# Using NNAPI and CoreML with ONNX Runtime Mobile - Usage of NNAPI on Android platforms is via the NNAPI Execution Provider (EP). - See the [NNAPI Execution Provider](../../execution-providers/NNAPI-ExecutionProvider.md) documentation for more details. @@ -19,7 +19,7 @@ If performing a custom build of ONNX Runtime, support for the NNAPI EP or CoreML ### Create a minimal build with NNAPI EP or CoreML EP support -Please see [the instructions](../../build/android-ios.md) for setting up the Android or iOS environment required to build. The Android build can be cross-compiled on Windows or Linux. The iOS/macOS build must be performed on a mac machine. +Please see [Build for Android](../../build/android.md) and [Build for iOS](../../build/ios.md) for setting up the Android or iOS environment required to build. The Android build can be cross-compiled on Windows or Linux. The iOS/macOS build must be performed on a mac machine. Once you have all the necessary components setup, follow the instructions to [create the custom build](./custom-build.md), with the following changes: - Replace `--minimal_build` with `--minimal_build extended` to enable support for execution providers that dynamically create kernels at runtime, which is required by the NNAPI EP and CoreML EP.