saymrwulf/onnxruntime
1
0
Fork 0
mirror of https://github.com/saymrwulf/onnxruntime.git synced 2026-07-16 18:31:27 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

initial docs for SNPE EP (#11705)

Browse source 
* SNPE EP docs

* minor corrections

* fixes

* c++

* remove

* Update docs/execution-providers/SNPE-ExecutionProvider.md

Co-authored-by: manashgoswami <magoswam@microsoft.com>

* review comments

Co-authored-by: manashgoswami <magoswam@microsoft.com>
This commit is contained in:
George Wu 2022-06-03 14:19:57 -07:00 committed by GitHub
parent b51e301eaa
commit 70a3468850
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
2 changed files with 97 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

29
docs/build/eps.md vendored
View file

@ -405,6 +405,35 @@ index 7dfa97c..6d99e71 100644
Dockerfile instructions are available [here](https://github.com/microsoft/onnxruntime/tree/master/dockerfiles#nuphar).
---
## SNPE
See more information on the SNPE execution provider [here](../execution-providers/SNPE-ExecutionProvider.md).
### Prerequisites
{: .no_toc }
* Qualcomm Neural Processing SDK [Linux/Android](https://developer.qualcomm.com/software/qualcomm-neural-processing-sdk)
* Qualcomm Neural Processing SDK [Windows](https://developer.qualcomm.com/software/qualcomm-neural-processing-sdk/windows-on-snapdragon)
### Build Instructions
{: .no_toc }
#### Windows
```
build.bat --use_snpe --snpe_root=[location-of-SNPE_SDK] --build_shared_lib --cmake_generator "Visual Studio 16 2019" --skip_submodule_sync --config Release --build_dir \build\Windows
```
#### Android (Cross-Compile):
Please reference [Build OnnxRuntime For Android](android.md)
```
# on Windows
build.bat --build_shared_lib --skip_submodule_sync --android --config Release --use_snpe --snpe_root [location-of-SNPE_SDK] --android_sdk_path [location-of-android_SDK] --android_ndk_path [location-of-android_NDK] --android_abi arm64-v8a --android_api [api-version] --cmake_generator Ninja --build_dir build\Android
# on Linux
build.sh --build_shared_lib --skip_submodule_sync --android --config Release --use_snpe --snpe_root [location-of-SNPE_SDK] --android_sdk_path [location-of-android_SDK] --android_ndk_path [location-of-android_NDK] --android_abi arm64-v8a --android_api [api-version] --cmake_generator Ninja --build_dir build/Android
```
---
## DirectML

68
docs/execution-providers/SNPE-ExecutionProvider.md Normal file
View file

@ -0,0 +1,68 @@
---
title: SNPE
description: Execute ONNX models with SNPE Execution Provider
parent: Execution Providers
nav_order: 1
redirect_from: /docs/reference/execution-providers/SNPE-ExecutionProvider
---
# SNPE Execution Provider
{: .no_toc }
The SNPE Execution Provider for ONNX Runtime enables hardware accelerated execution on Qualcomm Snapdragon CPU, the Qualcomm Adreno<sup>TM</sup> GPU, or the Hexagon DSP. This execution provider makes use of the Qualcomm Snapdragon Neural Processing Engine SDK.
This execution provider uses the AOT converted DLC code as an embedded node in the ONNX model file.
## Contents
{: .no_toc }
* TOC placeholder
{:toc}
## Install Pre-requisites
Download the SNPE toolkit from the Qualcomm Developer Network for [Android/Linux](https://developer.qualcomm.com/software/qualcomm-neural-processing-sdk)
or [Windows](https://developer.qualcomm.com/software/qualcomm-neural-processing-sdk/windows-on-snapdragon)
### SNPE Version Requirements
The SNPE version used with the ONNX Runtime SNPE Execution Provider must match the version used to generate the quantized SNPE-DLC file.
## Build
For build instructions, please see the [BUILD page](../build/eps.md#snpe).
## Configuration Options
The SNPE Execution Provider supports a number of options to set the SNPE Runtime configuration for executing the model. The `provider_option_keys`, `provider_options_values` and `num_keys` enable different options for the application. Each `provider_options_keys` accepts values as shown below:
|`provider_options_values` for `provider_options_keys = "runtime"`|Description|
|---|-----|
|CPU or CPU_FLOAT32|Using SnapDragon CPU with 32 bit data storage and math|
|DSP or DSP_FIXED8_TF|Using Hexagon DSP with 8bit fixed point Tensorflow style format data storage and 8bit fixed point Tensorflow style format math|
|GPU or GPU_FLOAT32_16_HYBRID|Using Adreno GPU with 16 bit data storage and 32 bit math|
|GPU_FLOAT16|GPU with 16 bit data storage and 16 bit math|
|AIP_FIXED_TF or AIP_FIXED8_TF|Using Snapdragon AIX+HVX with 8bit fixed point Tensorflow style format data storage and 8bit fixed point Tensorflow style format math|
|`provider_options_values` for `provider_options_keys = "buffer_type"`|Description|
|---|---|
|ITensor|Represents a tensor with n-dimensional data|
|TF8|User defined buffer with 8-bit quantized value|
|TF16|User defined buffer with 16-bit quantized value|
|UINT8|User defined buffer with unsigned int value|
|FLOAT|User defined buffer with float value|
## Usage
### C++
```
Ort::Env env = Ort::Env{ORT_LOGGING_LEVEL_ERROR, "Default"};
std::vector<const char*> snpe_option_keys;
std::vector<const char*> snpe_option_values;
snpe_option_keys.push_back("runtime");
snpe_option_values.push_back("DSP");
snpe_option_keys.push_back("buffer_type");
snpe_option_values.push_back("FLOAT");
Ort::SessionOptions session_options;
session_options.AppendExecutionProvider_SNPE(snpe_option_keys.data(), snpe_option_values.data(), snpe_option_keys.size());
Ort::Session session(env, model_path, session_options);
```
The C API details are [here](../get-started/with-c.md).