From 70a3468850fdc61dc34bd03a8e2eb4ed4f3228fe Mon Sep 17 00:00:00 2001 From: George Wu Date: Fri, 3 Jun 2022 14:19:57 -0700 Subject: [PATCH] initial docs for SNPE EP (#11705) * SNPE EP docs * minor corrections * fixes * c++ * remove * Update docs/execution-providers/SNPE-ExecutionProvider.md Co-authored-by: manashgoswami * review comments Co-authored-by: manashgoswami --- docs/build/eps.md | 29 ++++++++ .../SNPE-ExecutionProvider.md | 68 +++++++++++++++++++ 2 files changed, 97 insertions(+) create mode 100644 docs/execution-providers/SNPE-ExecutionProvider.md diff --git a/docs/build/eps.md b/docs/build/eps.md index 3d35342d24..838a4afee4 100644 --- a/docs/build/eps.md +++ b/docs/build/eps.md @@ -405,6 +405,35 @@ index 7dfa97c..6d99e71 100644 Dockerfile instructions are available [here](https://github.com/microsoft/onnxruntime/tree/master/dockerfiles#nuphar). +--- + +## SNPE +See more information on the SNPE execution provider [here](../execution-providers/SNPE-ExecutionProvider.md). + +### Prerequisites +{: .no_toc } +* Qualcomm Neural Processing SDK [Linux/Android](https://developer.qualcomm.com/software/qualcomm-neural-processing-sdk) +* Qualcomm Neural Processing SDK [Windows](https://developer.qualcomm.com/software/qualcomm-neural-processing-sdk/windows-on-snapdragon) + +### Build Instructions +{: .no_toc } + +#### Windows +``` +build.bat --use_snpe --snpe_root=[location-of-SNPE_SDK] --build_shared_lib --cmake_generator "Visual Studio 16 2019" --skip_submodule_sync --config Release --build_dir \build\Windows +``` +#### Android (Cross-Compile): + +Please reference [Build OnnxRuntime For Android](android.md) +``` +# on Windows +build.bat --build_shared_lib --skip_submodule_sync --android --config Release --use_snpe --snpe_root [location-of-SNPE_SDK] --android_sdk_path [location-of-android_SDK] --android_ndk_path [location-of-android_NDK] --android_abi arm64-v8a --android_api [api-version] --cmake_generator Ninja --build_dir build\Android + +# on Linux +build.sh --build_shared_lib --skip_submodule_sync --android --config Release --use_snpe --snpe_root [location-of-SNPE_SDK] --android_sdk_path [location-of-android_SDK] --android_ndk_path [location-of-android_NDK] --android_abi arm64-v8a --android_api [api-version] --cmake_generator Ninja --build_dir build/Android + +``` + --- ## DirectML diff --git a/docs/execution-providers/SNPE-ExecutionProvider.md b/docs/execution-providers/SNPE-ExecutionProvider.md new file mode 100644 index 0000000000..8eae6ccd77 --- /dev/null +++ b/docs/execution-providers/SNPE-ExecutionProvider.md @@ -0,0 +1,68 @@ +--- +title: SNPE +description: Execute ONNX models with SNPE Execution Provider +parent: Execution Providers +nav_order: 1 +redirect_from: /docs/reference/execution-providers/SNPE-ExecutionProvider +--- + +# SNPE Execution Provider +{: .no_toc } + +The SNPE Execution Provider for ONNX Runtime enables hardware accelerated execution on Qualcomm Snapdragon CPU, the Qualcomm AdrenoTM GPU, or the Hexagon DSP. This execution provider makes use of the Qualcomm Snapdragon Neural Processing Engine SDK. + +This execution provider uses the AOT converted DLC code as an embedded node in the ONNX model file. + +## Contents +{: .no_toc } + +* TOC placeholder +{:toc} + +## Install Pre-requisites + +Download the SNPE toolkit from the Qualcomm Developer Network for [Android/Linux](https://developer.qualcomm.com/software/qualcomm-neural-processing-sdk) +or [Windows](https://developer.qualcomm.com/software/qualcomm-neural-processing-sdk/windows-on-snapdragon) + +### SNPE Version Requirements + +The SNPE version used with the ONNX Runtime SNPE Execution Provider must match the version used to generate the quantized SNPE-DLC file. + +## Build +For build instructions, please see the [BUILD page](../build/eps.md#snpe). + +## Configuration Options +The SNPE Execution Provider supports a number of options to set the SNPE Runtime configuration for executing the model. The `provider_option_keys`, `provider_options_values` and `num_keys` enable different options for the application. Each `provider_options_keys` accepts values as shown below: + +|`provider_options_values` for `provider_options_keys = "runtime"`|Description| +|---|-----| +|CPU or CPU_FLOAT32|Using SnapDragon CPU with 32 bit data storage and math| +|DSP or DSP_FIXED8_TF|Using Hexagon DSP with 8bit fixed point Tensorflow style format data storage and 8bit fixed point Tensorflow style format math| +|GPU or GPU_FLOAT32_16_HYBRID|Using Adreno GPU with 16 bit data storage and 32 bit math| +|GPU_FLOAT16|GPU with 16 bit data storage and 16 bit math| +|AIP_FIXED_TF or AIP_FIXED8_TF|Using Snapdragon AIX+HVX with 8bit fixed point Tensorflow style format data storage and 8bit fixed point Tensorflow style format math| + +|`provider_options_values` for `provider_options_keys = "buffer_type"`|Description| +|---|---| +|ITensor|Represents a tensor with n-dimensional data| +|TF8|User defined buffer with 8-bit quantized value| +|TF16|User defined buffer with 16-bit quantized value| +|UINT8|User defined buffer with unsigned int value| +|FLOAT|User defined buffer with float value| + +## Usage +### C++ +``` +Ort::Env env = Ort::Env{ORT_LOGGING_LEVEL_ERROR, "Default"}; +std::vector snpe_option_keys; +std::vector snpe_option_values; +snpe_option_keys.push_back("runtime"); +snpe_option_values.push_back("DSP"); +snpe_option_keys.push_back("buffer_type"); +snpe_option_values.push_back("FLOAT"); +Ort::SessionOptions session_options; +session_options.AppendExecutionProvider_SNPE(snpe_option_keys.data(), snpe_option_values.data(), snpe_option_keys.size()); +Ort::Session session(env, model_path, session_options); +``` + +The C API details are [here](../get-started/with-c.md).