mirror of
https://github.com/saymrwulf/onnxruntime.git
synced 2026-07-29 20:14:01 +00:00
SEO audit of docs website (#9780)
This commit is contained in:
Nat Kershaw (MSFT)
GitHub
parent
a5c0ba8cf3
commit
91e029d4f3
34 changed files with 202 additions and 129 deletions
|
|
@ -8,6 +8,8 @@ redirect_from: /docs/reference/api/csharp-api
|
|||
# C# API Reference
|
||||
{: .no_toc }
|
||||
|
||||
This page shows the main elements of the C# API for ONNX Runtime.
|
||||
|
||||
### OrtEnv
|
||||
```cs
|
||||
class OrtEnv
|
||||
|
|
|
|||
107
docs/build/android-ios.md → docs/build/android.md
vendored
107
docs/build/android-ios.md → docs/build/android.md
vendored
|
|
@ -1,13 +1,15 @@
|
|||
---
|
||||
title: Build for Android/iOS
|
||||
parent: Build ORT
|
||||
title: Build for Android
|
||||
parent: Build ONNX Runtime
|
||||
nav_order: 5
|
||||
redirect_from: /docs/how-to/build/android-ios,/docs/build/android-ios
|
||||
---
|
||||
`
|
||||
# Build ONNX Runtime for Android and iOS
|
||||
|
||||
# Build ONNX Runtime for Android
|
||||
{: .no_toc }
|
||||
|
||||
Below are general build instructions for Android and iOS. For examples of deploying ONNX Runtime on mobile platforms (includes overall smaller package size and other configurations), see [Mobile Tutorials](../tutorials/mobile).
|
||||
Follow the instructions below to build ONNX Runtime for Android.
|
||||
|
||||
|
||||
## Contents
|
||||
{: .no_toc }
|
||||
|
|
@ -15,9 +17,7 @@ Below are general build instructions for Android and iOS. For examples of deploy
|
|||
* TOC placeholder
|
||||
{:toc}
|
||||
|
||||
## Android
|
||||
|
||||
### Prerequisites
|
||||
## Prerequisites
|
||||
|
||||
The SDK and NDK packages can be installed via Android Studio or the sdkmanager command line tool.
|
||||
|
||||
|
|
@ -29,7 +29,7 @@ Resources:
|
|||
- [Android ABIs](https://developer.android.com/ndk/guides/abis)
|
||||
- [System Images](https://developer.android.com/topic/generic-system-image)
|
||||
|
||||
#### Android Studio
|
||||
### Android Studio
|
||||
|
||||
1. [Install](https://developer.android.com/studio) Android Studio
|
||||
|
||||
|
|
@ -45,7 +45,7 @@ Resources:
|
|||
* The NDK path will be the 'ndk/{version}' subdirectory of the SDK path shown
|
||||
* e.g. if 21.1.6352462 is installed it will be {SDK path}/ndk/21.1.6352462
|
||||
|
||||
#### sdkmanager from command line tools
|
||||
### sdkmanager from command line tools
|
||||
|
||||
* If necessary install the Java Runtime Environment and set the JAVA_HOME environment variable to point to it
|
||||
* https://www.java.com/en/download/
|
||||
|
|
@ -83,9 +83,9 @@ Resources:
|
|||
* NDK path in our example with this install would be `.../Android/ndk/21.1.6352462`
|
||||
* NOTE: If you install the ndk-bundle package the path will be `.../Android/ndk-bundle` as there's no version number
|
||||
|
||||
### Android Build Instructions
|
||||
## Android Build Instructions
|
||||
|
||||
#### Cross compiling on Windows
|
||||
### Cross compiling on Windows
|
||||
|
||||
The [Ninja](https://ninja-build.org/) generator needs to be used to build on Windows as the Visual Studio generator doesn't support Android.
|
||||
|
||||
|
|
@ -99,13 +99,13 @@ e.g. using the paths from our example
|
|||
./build.bat --android --android_sdk_path .../Android --android_ndk_path .../Android/ndk/21.1.6352462 --android_abi arm64-v8a --android_api 27 --cmake_generator Ninja
|
||||
```
|
||||
|
||||
#### Cross compiling on Linux and macOS
|
||||
### Cross compiling on Linux and macOS
|
||||
|
||||
```bash
|
||||
./build.sh --android --android_sdk_path <android sdk path> --android_ndk_path <android ndk path> --android_abi <android abi, e.g., arm64-v8a (default) or armeabi-v7a> --android_api <android api level, e.g., 27 (default)>
|
||||
```
|
||||
|
||||
#### Build Android Archive (AAR)
|
||||
### Build Android Archive (AAR)
|
||||
|
||||
Android Archive (AAR) files, which can be imported directly in Android Studio, will be generated in your_build_dir/java/build/android/outputs/aar, by using the above building commands with `--build_java`
|
||||
|
||||
|
|
@ -119,86 +119,11 @@ To build on Windows with `--build_java` enabled you must also:
|
|||
* run the build from an admin window
|
||||
* the Java build needs permissions to create a symlink, which requires an admin window
|
||||
|
||||
### Android NNAPI Execution Provider
|
||||
## Android NNAPI Execution Provider
|
||||
|
||||
If you want to use NNAPI Execution Provider on Android, see [NNAPI Execution Provider](../execution-providers/NNAPI-ExecutionProvider).
|
||||
|
||||
#### Build Instructions
|
||||
### Build Instructions
|
||||
|
||||
Android NNAPI Execution Provider can be built using building commands in [Android Build instructions](#android-build-instructions) with `--use_nnapi`
|
||||
|
||||
---
|
||||
|
||||
## iOS
|
||||
|
||||
### Prerequisites
|
||||
|
||||
* A Mac computer with latest macOS
|
||||
* Xcode, https://developer.apple.com/xcode/
|
||||
* CMake, https://cmake.org/download/
|
||||
* Python 3, https://www.python.org/downloads/mac-osx/
|
||||
|
||||
### General Info
|
||||
|
||||
* iOS Platforms
|
||||
|
||||
The following two platforms are supported
|
||||
* iOS device (iPhone, iPad) with arm64 architecture
|
||||
* iOS simulator with x86_64 architecture
|
||||
|
||||
The following platforms are *not* supported
|
||||
* armv7
|
||||
* armv7s
|
||||
* i386 architectures
|
||||
* tvOS
|
||||
* watchOS platforms are not currently supported.
|
||||
|
||||
* apple_deploy_target
|
||||
|
||||
Specify the minimum version of the target platform (iOS) on which the target binaries are to be deployed.
|
||||
|
||||
* Code Signing
|
||||
|
||||
If the code signing development team ID or code signing identity is specified, and has a valid code signing certificate, Xcode will code sign the onnxruntime library in the building process. Otherwise, the onnxruntime will be built without code signing. It may be required or desired to code sign the library for iOS devices. For more information, see [Code Signing](https://developer.apple.com/support/code-signing/).
|
||||
|
||||
### Build Instructions
|
||||
|
||||
Run one of the following build scripts from the ONNX Runtime repository root:
|
||||
|
||||
#### Cross build for iOS simulator
|
||||
|
||||
```bash
|
||||
./build.sh --config <Release|Debug|RelWithDebInfo|MinSizeRel> --use_xcode \
|
||||
--ios --ios_sysroot iphonesimulator --osx_arch x86_64 --apple_deploy_target <minimal iOS version>
|
||||
```
|
||||
|
||||
#### Cross build for iOS device
|
||||
|
||||
```bash
|
||||
./build.sh --config <Release|Debug|RelWithDebInfo|MinSizeRel> --use_xcode \
|
||||
--ios --ios_sysroot iphoneos --osx_arch arm64 --apple_deploy_target <minimal iOS version>
|
||||
```
|
||||
|
||||
#### Cross build for iOS device and code sign the library using development team ID
|
||||
|
||||
```bash
|
||||
./build.sh --config <Release|Debug|RelWithDebInfo|MinSizeRel> --use_xcode \
|
||||
--ios --ios_sysroot iphoneos --osx_arch arm64 --apple_deploy_target <minimal iOS version> \
|
||||
--xcode_code_signing_team_id <Your Apple developmemt team ID>
|
||||
```
|
||||
|
||||
#### Cross build for iOS device and code sign the library using code sign identity
|
||||
|
||||
```bash
|
||||
./build.sh --config <Release|Debug|RelWithDebInfo|MinSizeRel> --use_xcode \
|
||||
--ios --ios_sysroot iphoneos --osx_arch arm64 --apple_deploy_target <minimal iOS version> \
|
||||
--xcode_code_signing_identity <Your preferred code sign identity>
|
||||
```
|
||||
|
||||
### CoreML Execution Provider
|
||||
|
||||
If you want to use CoreML Execution Provider on iOS or macOS, see [CoreML Execution Provider](../execution-providers/CoreML-ExecutionProvider).
|
||||
|
||||
#### Build Instructions
|
||||
|
||||
CoreML Execution Provider can be built using building commands in [iOS Build instructions](#build-instructions-1) with `--use_coreml`
|
||||
4
docs/build/eps.md
vendored
4
docs/build/eps.md
vendored
|
|
@ -1,7 +1,9 @@
|
|||
---
|
||||
title: Build with different EPs
|
||||
parent: Build ORT
|
||||
parent: Build ONNX Runtime
|
||||
description: Learm how to build ONNX Runtime from source for different execution providers
|
||||
nav_order: 3
|
||||
redirect_from: /docs/how-to/build/eps
|
||||
---
|
||||
|
||||
# Build ONNX Runtime with Execution Providers
|
||||
|
|
|
|||
5
docs/build/index.md
vendored
5
docs/build/index.md
vendored
|
|
@ -1,8 +1,11 @@
|
|||
---
|
||||
title: Build ORT
|
||||
title: Build ONNX Runtime
|
||||
has_children: true
|
||||
nav_order: 5
|
||||
redirect_from: /docs/how-to/build
|
||||
---
|
||||
|
||||
# Build ONNX Runtime from source
|
||||
{: .no_toc }
|
||||
|
||||
For production deployments, it's strongly recommended to build only from an [official release branch](https://github.com/microsoft/onnxruntime/releases).
|
||||
|
|
|
|||
9
docs/build/inferencing.md
vendored
9
docs/build/inferencing.md
vendored
|
|
@ -1,12 +1,15 @@
|
|||
---
|
||||
title: Build for inferencing
|
||||
parent: Build ORT
|
||||
parent: Build ONNX Runtime
|
||||
nav_order: 1
|
||||
redirect_from: /docs/how-to/build/inferencing
|
||||
---
|
||||
|
||||
# Build ONNX Runtime for inferencing
|
||||
{: .no_toc }
|
||||
|
||||
Follow the instructions below to build ONNX Runtime to perform inference.
|
||||
|
||||
## Contents
|
||||
{: .no_toc }
|
||||
|
||||
|
|
@ -512,8 +515,8 @@ ls -l /code/onnxruntime/build/Linux/MinSizeRel/dist/*.whl
|
|||
|
||||
---
|
||||
|
||||
### Android/iOS
|
||||
Please see [Build for Android/iOS](./android-ios.md)
|
||||
### Mobile
|
||||
Please see [Build for Android](./android.md) and [Build for iOS](./ios.md)
|
||||
|
||||
### Web
|
||||
Please see [Build for Web](./web.md)
|
||||
|
|
|
|||
90
docs/build/ios.md
vendored
Normal file
90
docs/build/ios.md
vendored
Normal file
|
|
@ -0,0 +1,90 @@
|
|||
---
|
||||
title: Build for iOS
|
||||
parent: Build ONNX Runtime
|
||||
nav_order: 5
|
||||
---
|
||||
|
||||
# Build ONNX Runtime for iOS
|
||||
{: .no_toc }
|
||||
|
||||
Follow the instructions below to build ONNX Runtime for iOS.
|
||||
|
||||
|
||||
## Contents
|
||||
{: .no_toc }
|
||||
|
||||
* TOC placeholder
|
||||
{:toc}
|
||||
|
||||
## General Info
|
||||
|
||||
* iOS Platforms
|
||||
|
||||
The following two platforms are supported
|
||||
* iOS device (iPhone, iPad) with arm64 architecture
|
||||
* iOS simulator with x86_64 architecture
|
||||
|
||||
The following platforms are *not* supported
|
||||
* armv7
|
||||
* armv7s
|
||||
* i386 architectures
|
||||
* tvOS
|
||||
* watchOS platforms are not currently supported.
|
||||
|
||||
* apple_deploy_target
|
||||
|
||||
Specify the minimum version of the target platform (iOS) on which the target binaries are to be deployed.
|
||||
|
||||
* Code Signing
|
||||
|
||||
If the code signing development team ID or code signing identity is specified, and has a valid code signing certificate, Xcode will code sign the onnxruntime library in the building process. Otherwise, the onnxruntime will be built without code signing. It may be required or desired to code sign the library for iOS devices. For more information, see [Code Signing](https://developer.apple.com/support/code-signing/).
|
||||
|
||||
|
||||
## Prerequisites
|
||||
|
||||
* A Mac computer with latest macOS
|
||||
* Xcode, https://developer.apple.com/xcode/
|
||||
* CMake, https://cmake.org/download/
|
||||
* Python 3, https://www.python.org/downloads/mac-osx/
|
||||
|
||||
## Build Instructions
|
||||
|
||||
Run one of the following build scripts from the ONNX Runtime repository root:
|
||||
|
||||
### Cross compile for iOS simulator
|
||||
|
||||
```bash
|
||||
./build.sh --config <Release|Debug|RelWithDebInfo|MinSizeRel> --use_xcode \
|
||||
--ios --ios_sysroot iphonesimulator --osx_arch x86_64 --apple_deploy_target <minimal iOS version>
|
||||
```
|
||||
|
||||
### Cross compile for iOS device
|
||||
|
||||
```bash
|
||||
./build.sh --config <Release|Debug|RelWithDebInfo|MinSizeRel> --use_xcode \
|
||||
--ios --ios_sysroot iphoneos --osx_arch arm64 --apple_deploy_target <minimal iOS version>
|
||||
```
|
||||
|
||||
### Cross compile for iOS device and code sign the library using development team ID
|
||||
|
||||
```bash
|
||||
./build.sh --config <Release|Debug|RelWithDebInfo|MinSizeRel> --use_xcode \
|
||||
--ios --ios_sysroot iphoneos --osx_arch arm64 --apple_deploy_target <minimal iOS version> \
|
||||
--xcode_code_signing_team_id <Your Apple developmemt team ID>
|
||||
```
|
||||
|
||||
### Cross compile for iOS device and code sign the library using code sign identity
|
||||
|
||||
```bash
|
||||
./build.sh --config <Release|Debug|RelWithDebInfo|MinSizeRel> --use_xcode \
|
||||
--ios --ios_sysroot iphoneos --osx_arch arm64 --apple_deploy_target <minimal iOS version> \
|
||||
--xcode_code_signing_identity <Your preferred code sign identity>
|
||||
```
|
||||
|
||||
### CoreML Execution Provider
|
||||
|
||||
If you want to use CoreML Execution Provider on iOS or macOS, see [CoreML Execution Provider](../execution-providers/CoreML-ExecutionProvider).
|
||||
|
||||
#### Build Instructions
|
||||
|
||||
CoreML Execution Provider can be built using building commands in [iOS Build instructions](#build-instructions-1) with `--use_coreml`
|
||||
7
docs/build/reduced.md
vendored
7
docs/build/reduced.md
vendored
|
|
@ -1,15 +1,16 @@
|
|||
---
|
||||
title: Build with reduced size
|
||||
parent: Build ORT
|
||||
parent: Build ONNX Runtime
|
||||
nav_order: 6
|
||||
redirect_from: /docs/how-to/build/reduced
|
||||
---
|
||||
|
||||
# Build ORT with reduced size
|
||||
# Reduce the size of the ONNX Runtime build
|
||||
{: .no_toc }
|
||||
|
||||
For applications where package binary size is important, ONNX Runtime provides options to reduce the build size with some functional trade-offs.
|
||||
|
||||
To reduce the compiled binary size of ONNX Runtime, the operator kernels included in the build can be reduced to just the kernels required by your model/s.
|
||||
To reduce the compiled binary size of ONNX Runtime, the build can be configured to only include the operator kernels needed by your model(s).
|
||||
|
||||
For deployment on mobile devices specifically, please read more detailed guidance on [Deploy ONNX Runtime Mobile](../tutorials/mobile/).
|
||||
|
||||
|
|
|
|||
4
docs/build/training.md
vendored
4
docs/build/training.md
vendored
|
|
@ -1,7 +1,9 @@
|
|||
---
|
||||
title: Build for training
|
||||
parent: Build ORT
|
||||
parent: Build ONNX Runtime
|
||||
description: Learn how to build ONNX Runtime for training from source for different hardware targets
|
||||
nav_order: 2
|
||||
redirect_from: /docs/how-to/build/training
|
||||
---
|
||||
|
||||
# Build ONNX Runtime for training
|
||||
|
|
|
|||
4
docs/build/web.md
vendored
4
docs/build/web.md
vendored
|
|
@ -1,7 +1,9 @@
|
|||
---
|
||||
title: Build for web
|
||||
parent: Build ORT
|
||||
parent: Build ONNX Runtime
|
||||
description: Learn how to build ONNX Runtime from source to deploy on the web
|
||||
nav_order: 4
|
||||
redirect_from: /docs/how-to/build/web
|
||||
---
|
||||
|
||||
# Build ONNX Runtime for Web
|
||||
|
|
|
|||
|
|
@ -1,6 +1,8 @@
|
|||
---
|
||||
title: ORT Ecosystem
|
||||
title: ONNX Runtime Ecosystem
|
||||
description: See examples of how ONNX Runtime working end to end within the Azure AI and ML landscape and ecosystem
|
||||
nav_order: 11
|
||||
redirect_from: /docs/tutorials/ecosystem
|
||||
---
|
||||
# ORT Ecosystem
|
||||
{: .no_toc }
|
||||
|
|
|
|||
|
|
@ -1,13 +1,15 @@
|
|||
---
|
||||
title: ARM Compute Library
|
||||
title: ARM Compute Library (ACL)
|
||||
description: Instructions to execute ONNX Runtime with the ACL Execution Provider
|
||||
parent: Execution Providers
|
||||
nav_order: 4
|
||||
redirect_from: /docs/reference/execution-providers/ACL-ExecutionProvider
|
||||
---
|
||||
|
||||
# ACL Execution Provider
|
||||
{: .no_toc }
|
||||
|
||||
[Arm Compute Library](https://github.com/ARM-software/ComputeLibrary){:target="_blank"} is an open source inference engine maintained by Arm and Linaro companies. The integration of ACL as an execution provider (EP) into ONNX Runtime accelerates performance of ONNX model workloads across Armv8 cores.
|
||||
The integration of ACL as an execution provider (EP) into ONNX Runtime accelerates performance of ONNX model workloads across Armv8 cores. [Arm Compute Library](https://github.com/ARM-software/ComputeLibrary){:target="_blank"} is an open source inference engine maintained by Arm and Linaro companies.
|
||||
|
||||
|
||||
## Contents
|
||||
|
|
|
|||
|
|
@ -1,7 +1,9 @@
|
|||
---
|
||||
title: ARM NN
|
||||
title: Arm NN
|
||||
description: Instructions to execute ONNX Runtime with Arm NN on Armv8 cores
|
||||
parent: Execution Providers
|
||||
nav_order: 3
|
||||
redirect_from: /docs/reference/execution-providers/ArmNN-ExecutionProvider
|
||||
---
|
||||
|
||||
# ArmNN Execution Provider
|
||||
|
|
@ -13,7 +15,7 @@ nav_order: 3
|
|||
* TOC placeholder
|
||||
{:toc}
|
||||
|
||||
[ArmNN](https://github.com/ARM-software/armnn) is an open source inference engine maintained by Arm and Linaro companies. The integration of ArmNN as an execution provider (EP) into ONNX Runtime accelerates performance of ONNX model workloads across Armv8 cores.
|
||||
Accelerate performance of ONNX model workloads across Armv8 cores with the ArmNN execution provider. [ArmNN](https://github.com/ARM-software/armnn) is an open source inference engine maintained by Arm and Linaro companies.
|
||||
|
||||
## Build
|
||||
For build instructions, please see the [BUILD page](../build/eps.md#armnn).
|
||||
|
|
|
|||
|
|
@ -1,7 +1,9 @@
|
|||
---
|
||||
title: CUDA
|
||||
description: Instructions to execute ONNX Runtime applications with CUDA
|
||||
parent: Execution Providers
|
||||
nav_order: 1
|
||||
redirect_from: /docs/reference/execution-providers/CUDA-ExecutionProvider
|
||||
---
|
||||
|
||||
# CUDA Execution Provider
|
||||
|
|
|
|||
|
|
@ -1,7 +1,9 @@
|
|||
---
|
||||
title: CoreML
|
||||
description: Instructions to execute ONNX Runtime with CoreML
|
||||
parent: Execution Providers
|
||||
nav_order: 2
|
||||
redirect_from: /docs/reference/execution-providers/CoreML-ExecutionProvider
|
||||
---
|
||||
{::options toc_levels="2" /}
|
||||
|
||||
|
|
@ -29,7 +31,7 @@ Pre-built binaries of ONNX Runtime Mobile with CoreML EP for iOS are published t
|
|||
|
||||
## Build
|
||||
|
||||
For build instructions for iOS devices, please see [How to: Build for Android/iOS](../build/android-ios.md#coreml-execution-provider).
|
||||
For build instructions for iOS devices, please see [Build for iOS](../build/ios.md#coreml-execution-provider).
|
||||
|
||||
## Usage
|
||||
|
||||
|
|
|
|||
|
|
@ -1,18 +1,20 @@
|
|||
---
|
||||
title: Direct ML
|
||||
description: Instructions to execute ONNX Runtime with the DirectML execution provider
|
||||
parent: Execution Providers
|
||||
nav_order: 5
|
||||
redirect_from: /docs/reference/execution-providers/DirectML-ExecutionProvider
|
||||
---
|
||||
|
||||
# DirectML Execution Provider
|
||||
{: .no_toc }
|
||||
|
||||
The DirectML Execution Provider is a component of ONNX Runtime that uses DirectML to accelerate inference of ONNX models. The DirectML execution provider is capable of greatly improving evaluation time of models using commodity GPU hardware, without sacrificing broad hardware support or requiring vendor-specific extensions to be installed.
|
||||
|
||||
DirectML is a high-performance, hardware-accelerated DirectX 12 library for machine learning on Windows. DirectML provides GPU acceleration for common machine learning tasks across a broad range of supported hardware and drivers.
|
||||
|
||||
When used standalone, the DirectML API is a low-level DirectX 12 library and is suitable for high-performance, low-latency applications such as frameworks, games, and other real-time applications. The seamless interoperability of DirectML with Direct3D 12 as well as its low overhead and conformance across hardware makes DirectML ideal for accelerating machine learning when both high performance is desired, and the reliability and predictability of results across hardware is critical.
|
||||
|
||||
The *DirectML Execution Provider* is an optional component of ONNX Runtime that uses DirectML to accelerate inference of ONNX models. The DirectML execution provider is capable of greatly improving evaluation time of models using commodity GPU hardware, without sacrificing broad hardware support or requiring vendor-specific extensions to be installed.
|
||||
|
||||
The DirectML Execution Provider currently uses DirectML version 1.4.2.
|
||||
|
||||
## Contents
|
||||
|
|
|
|||
|
|
@ -1,7 +1,9 @@
|
|||
---
|
||||
title: AMD MI GraphX
|
||||
title: AMD MIGraphX
|
||||
description: Instructions to execute ONNX Runtime with the AMD MIGraphX execution provider
|
||||
parent: Execution Providers
|
||||
nav_order: 6
|
||||
redirect_from: /docs/reference/execution-providers/MIGraphX-ExecutionProvider
|
||||
---
|
||||
|
||||
# MIGraphX Execution Provider
|
||||
|
|
|
|||
|
|
@ -1,13 +1,15 @@
|
|||
---
|
||||
title: NNAPI
|
||||
description: Instructions to execute ONNX Runtime with the NNAPI execution provider
|
||||
parent: Execution Providers
|
||||
nav_order: 7
|
||||
redirect_from: /docs/reference/execution-providers/NNAPI-ExecutionProvider
|
||||
---
|
||||
{::options toc_levels="2" /}
|
||||
|
||||
# NNAPI Execution Provider
|
||||
|
||||
[Android Neural Networks API (NNAPI)](https://developer.android.com/ndk/guides/neuralnetworks) is a unified interface to CPU, GPU, and NN accelerators on Android.
|
||||
Accelerate ONNX models on Android devices with ONNX Runtime and the NNAPI execution provider. [Android Neural Networks API (NNAPI)](https://developer.android.com/ndk/guides/neuralnetworks) is a unified interface to CPU, GPU, and NN accelerators on Android.
|
||||
|
||||
## Contents
|
||||
{: .no_toc }
|
||||
|
|
|
|||
|
|
@ -1,15 +1,19 @@
|
|||
---
|
||||
title: NUPHAR
|
||||
description: Instructions to execute ONNX Runtime with the Nuphar execution provider
|
||||
parent: Execution Providers
|
||||
nav_order: 8
|
||||
redirect_from: /docs/reference/execution-providers/Nuphar-ExecutionProvider
|
||||
---
|
||||
|
||||
# Nuphar Execution Provider
|
||||
{: .no_toc }
|
||||
|
||||
Tap into the power of Nuphar through ONNX Runtime to accelerate inferencing of ONNX models.
|
||||
|
||||
NUPHAR stands for Neural-network Unified Preprocessing Heterogeneous Architecture. As an execution provider in the ONNX Runtime, it is built on top of [TVM](https://github.com/dmlc/tvm) and [LLVM](https://llvm.org) to accelerate ONNX models by compiling nodes in subgraphs into optimized functions via JIT. It also provides JIT caching to save compilation time at runtime.
|
||||
|
||||
Developers can tap into the power of Nuphar through ONNX Runtime to accelerate inferencing of ONNX models. The Nuphar execution provider comes with a common ONNX to TVM lowering [library](https://github.com/microsoft/onnxruntime/tree/master/onnxruntime/core/codegen) that can potentially be reused by other execution providers to leverage TVM. With the Nuphar execution provider, the ONNX Runtime delivers better inferencing performance on the same hardware compared to generic X64 CPU acceleration, especially for quantized recurrent neural networks. Various products at Microsoft have seen up to a 5x improvement in performance with no loss of accuracy, by running quantized LSTMs via the Nuphar execution provider in the ONNX Runtime.
|
||||
The Nuphar execution provider comes with a common ONNX to TVM lowering [library](https://github.com/microsoft/onnxruntime/tree/master/onnxruntime/core/codegen) that can potentially be reused by other execution providers to leverage TVM. With the Nuphar execution provider, the ONNX Runtime delivers better inferencing performance on the same hardware compared to generic X64 CPU acceleration, especially for quantized recurrent neural networks. Various products at Microsoft have seen up to a 5x improvement in performance with no loss of accuracy, by running quantized LSTMs via the Nuphar execution provider in the ONNX Runtime.
|
||||
|
||||
## Contents
|
||||
{: .no_toc }
|
||||
|
|
|
|||
|
|
@ -1,13 +1,15 @@
|
|||
---
|
||||
title: OpenVINO
|
||||
description: Instructions to execute ONNX Runtime with the Intel OpenVINO execution provider
|
||||
parent: Execution Providers
|
||||
nav_order: 10
|
||||
redirect_from: /docs/reference/execution-providers/OpenVINO-ExecutionProvider
|
||||
---
|
||||
|
||||
# OpenVINO Execution Provider
|
||||
{: .no_toc }
|
||||
|
||||
OpenVINO Execution Provider enables deep learning inference on Intel CPUs, Intel integrated GPUs and Intel<sup>®</sup> Movidius<sup>TM</sup> Vision Processing Units (VPUs). Please refer to [this](https://software.intel.com/en-us/openvino-toolkit/hardware) page for details on the Intel hardware supported.
|
||||
Accelerate ONNX models on Intel CPUs, GPUs and VPUs with ONNX Runtime and the Intel OpenVINO execution provider. Please refer to [this](https://software.intel.com/en-us/openvino-toolkit/hardware) page for details on the Intel hardware supported.
|
||||
|
||||
## Contents
|
||||
{: .no_toc }
|
||||
|
|
|
|||
|
|
@ -1,7 +1,9 @@
|
|||
---
|
||||
title: RKNPU
|
||||
description: Instructions to execute ONNX Runtime on Rockchip NPUs with the RKNPU execution provider
|
||||
parent: Execution Providers
|
||||
nav_order: 11
|
||||
redirect_from: /docs/reference/execution-providers/RKNPU-ExecutionProvider
|
||||
---
|
||||
|
||||
# RKNPU Execution Provider
|
||||
|
|
|
|||
|
|
@ -1,16 +1,18 @@
|
|||
---
|
||||
title: TensorRT
|
||||
description: Instructions to execute ONNX Runtime on NVIDIA GPUs with the TensorRT execution provider
|
||||
parent: Execution Providers
|
||||
nav_order: 12
|
||||
redirect_from: /docs/reference/execution-providers/TensorRT-ExecutionProvider
|
||||
---
|
||||
|
||||
# TensorRT Execution Provider
|
||||
{: .no_toc }
|
||||
|
||||
The TensorRT execution provider in the ONNX Runtime makes use of NVIDIA's [TensorRT](https://developer.nvidia.com/tensorrt) Deep Learning inferencing engine to accelerate ONNX model in their family of GPUs. Microsoft and NVIDIA worked closely to integrate the TensorRT execution provider with ONNX Runtime.
|
||||
|
||||
With the TensorRT execution provider, the ONNX Runtime delivers better inferencing performance on the same hardware compared to generic GPU acceleration.
|
||||
|
||||
The TensorRT execution provider in the ONNX Runtime makes use of NVIDIA's [TensorRT](https://developer.nvidia.com/tensorrt) Deep Learning inferencing engine to accelerate ONNX model in their family of GPUs. Microsoft and NVIDIA worked closely to integrate the TensorRT execution provider with ONNX Runtime.
|
||||
|
||||
## Contents
|
||||
{: .no_toc }
|
||||
|
||||
|
|
|
|||
|
|
@ -1,7 +1,9 @@
|
|||
---
|
||||
title: Vitis AI
|
||||
description: Instructions to execute ONNX Runtime on Xilinx devices with the Vitis AI execution provider
|
||||
parent: Execution Providers
|
||||
nav_order: 13
|
||||
redirect_from: /docs/reference/execution-providers/Vitis-AI-ExecutionProvider
|
||||
---
|
||||
|
||||
# Vitis-AI Execution Provider
|
||||
|
|
|
|||
|
|
@ -1,7 +1,9 @@
|
|||
---
|
||||
title: Add a new execution provider
|
||||
description: Instructions to add a new execution provider to ONNX Runtime
|
||||
parent: Execution Providers
|
||||
nav_order: 14
|
||||
redirect_from: /docs/how-to/add-execution-provider
|
||||
---
|
||||
|
||||
# Add a new Execution Provider to ONNX Runtime
|
||||
|
|
|
|||
|
|
@ -1,17 +1,21 @@
|
|||
---
|
||||
title: Intel oneDNN
|
||||
description: Instructions to execute ONNX Runtime with the Intel oneDNN execution provider
|
||||
parent: Execution Providers
|
||||
nav_order: 9
|
||||
redirect_from: /docs/reference/execution-providers/oneDNN-ExecutionProvider
|
||||
---
|
||||
|
||||
# oneDNN Execution Provider
|
||||
{: .no_toc }
|
||||
|
||||
*Formerly "DNNL"*
|
||||
Accelerate performance of ONNX Runtime using Intel® Math Kernel Library for Deep Neural Networks (Intel® DNNL) optimized primitives with the Intel oneDNN execution provider.
|
||||
|
||||
Intel® oneAPI Deep Neural Network Library is an open-source performance library for deep-learning applications. The library accelerates deep-learning applications and frameworks on Intel® architecture and Intel® Processor Graphics Architecture. Intel DNNL contains vectorized and threaded building blocks that you can use to implement deep neural networks (DNN) with C and C++ interfaces.
|
||||
|
||||
Intel and Microsoft have developed the DNNL Execution Provider (EP) for ONNX Runtime to accelerate performance of ONNX Runtime using Intel® Math Kernel Library for Deep Neural Networks (Intel® DNNL) optimized primitives.
|
||||
The DNNL Execution Provider (EP) for ONNX Runtime is developed by a partnership between Intel and Microsoft.
|
||||
|
||||
*Formerly "DNNL"*
|
||||
|
||||
## Contents
|
||||
{: .no_toc }
|
||||
|
|
|
|||
|
|
@ -19,7 +19,7 @@ ONNX Runtime is an accelerator for machine learning models with multi platform s
|
|||
## Contribute and Customize
|
||||
|
||||
|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
|
||||
| <span class="fs-5"> [Build ORT Packages](./build){: .btn .mr-4 target="_blank"} </span> | <span class="fs-5"> [ORT GitHub](https://github.com/microsoft/onnxruntime){: .btn target="_blank"} </span> |
|
||||
| <span class="fs-5"> [Build ORT Packages](./build){: .btn .mr-4 target="_blank"} </span> | <span class="fs-5">ONNX Runtime GitHub](https://github.com/microsoft/onnxruntime){: .btn target="_blank"} </span> |
|
||||
|
||||
|
||||
---
|
||||
|
|
|
|||
|
|
@ -1,7 +1,9 @@
|
|||
---
|
||||
title: Install ORT
|
||||
title: Install ONNX Runtime
|
||||
description: Instructions to install ONNX Runtime on your target platform in your environment
|
||||
has_children: false
|
||||
nav_order: 1
|
||||
redirect_from: /docs/how-to/install
|
||||
---
|
||||
|
||||
# Install ONNX Runtime (ORT)
|
||||
|
|
|
|||
|
|
@ -1,7 +1,8 @@
|
|||
---
|
||||
title: Graph optimizations
|
||||
parent: Performance
|
||||
nav_order: 2
|
||||
nav_order: 3
|
||||
redirect_from: /docs/how-to/graph-optimizations
|
||||
---
|
||||
|
||||
# Graph Optimizations in ONNX Runtime
|
||||
|
|
|
|||
|
|
@ -1,13 +1,16 @@
|
|||
---
|
||||
title: ONNX Runtime Mobile Performance Tuning
|
||||
title: Tune Mobile Performance
|
||||
parent: Performance
|
||||
nav_order: 1
|
||||
nav_order: 2
|
||||
redirect_from: /docs/how-to/mobile-performance-tuning
|
||||
---
|
||||
{::options toc_levels="2" /}
|
||||
|
||||
# Performance Tuning
|
||||
# ONNX Runtime Mobile Performance Tuning
|
||||
|
||||
ONNX Runtime Mobile can be used to execute ORT format models using NNAPI (via the NNAPI Execution Provider (EP)) on Android platforms, and CoreML (via the CoreML EP) on iOS platforms. This section explains the details of how different optimizations affect performance, and provides some suggestions for performance testing with ORT format models.
|
||||
Learn how different optimizations affect performance, and get suggestions for performance testing with ORT format models.
|
||||
|
||||
ONNX Runtime Mobile can be used to execute ORT format models using NNAPI (via the NNAPI Execution Provider (EP)) on Android platforms, and CoreML (via the CoreML EP) on iOS platforms.
|
||||
|
||||
First, please review the introductory details in [using NNAPI and CoreML with ONNX Runtime Mobile](../tutorials/mobile/using-nnapi-coreml-with-ort-mobile.md).
|
||||
|
||||
|
|
|
|||
|
|
@ -2,6 +2,7 @@
|
|||
title: Quantize ONNX Models
|
||||
parent: Performance
|
||||
nav_order: 4
|
||||
redirect_from: /docs/how-to/quantization
|
||||
---
|
||||
# Quantize ONNX Models
|
||||
{: .no_toc }
|
||||
|
|
|
|||
|
|
@ -1,7 +1,8 @@
|
|||
---
|
||||
title: Tune performance
|
||||
parent: Performance
|
||||
nav_order: 3
|
||||
nav_order: 1
|
||||
redirect_from: /docs/how-to/tune-performance
|
||||
---
|
||||
|
||||
# ONNX Runtime Performance Tuning
|
||||
|
|
|
|||
|
|
@ -1,5 +1,6 @@
|
|||
---
|
||||
title: API Basics
|
||||
descriptions:
|
||||
parent: Tutorials
|
||||
nav_order: 1
|
||||
---
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ grand_parent: Tutorials
|
|||
nav_order: 7
|
||||
---
|
||||
|
||||
## Limitations
|
||||
# Limitations
|
||||
|
||||
A minimal build has the following limitations:
|
||||
- No support for ONNX format models
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@ nav_order: 3
|
|||
* TOC
|
||||
{:toc}
|
||||
|
||||
## Converting ONNX models to ORT format
|
||||
# Converting ONNX models to ORT format
|
||||
|
||||
Conversion of ONNX format models to ORT format utilizes the ONNX Runtime python package, as the model is loaded into ONNX Runtime and optimized as part of the conversion process.
|
||||
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ grand_parent: Tutorials
|
|||
nav_order: 6
|
||||
---
|
||||
|
||||
## Using NNAPI and CoreML with ONNX Runtime Mobile
|
||||
# Using NNAPI and CoreML with ONNX Runtime Mobile
|
||||
|
||||
- Usage of NNAPI on Android platforms is via the NNAPI Execution Provider (EP).
|
||||
- See the [NNAPI Execution Provider](../../execution-providers/NNAPI-ExecutionProvider.md) documentation for more details.
|
||||
|
|
@ -19,7 +19,7 @@ If performing a custom build of ONNX Runtime, support for the NNAPI EP or CoreML
|
|||
|
||||
### Create a minimal build with NNAPI EP or CoreML EP support
|
||||
|
||||
Please see [the instructions](../../build/android-ios.md) for setting up the Android or iOS environment required to build. The Android build can be cross-compiled on Windows or Linux. The iOS/macOS build must be performed on a mac machine.
|
||||
Please see [Build for Android](../../build/android.md) and [Build for iOS](../../build/ios.md) for setting up the Android or iOS environment required to build. The Android build can be cross-compiled on Windows or Linux. The iOS/macOS build must be performed on a mac machine.
|
||||
|
||||
Once you have all the necessary components setup, follow the instructions to [create the custom build](./custom-build.md), with the following changes:
|
||||
- Replace `--minimal_build` with `--minimal_build extended` to enable support for execution providers that dynamically create kernels at runtime, which is required by the NNAPI EP and CoreML EP.
|
||||
|
|
|
|||
Loading…
Reference in a new issue