saymrwulf/onnxruntime
1
0
Fork 0
mirror of https://github.com/saymrwulf/onnxruntime.git synced 2026-07-28 20:11:22 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Rename STVM to TVM ep (#10190)

Browse source
This commit is contained in:
manashgoswami 2022-01-05 14:06:35 -08:00 committed by GitHub
parent 66efb22cda
commit 7f0b170fd9
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
4 changed files with 27 additions and 25 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

32
docs/execution-providers/STVM-ExecutionProvider.md → docs/execution-providers/TVM-ExecutionProvider.md
View file

@ -1,15 +1,17 @@
---
title: Standalone TVM (STVM)
description: Instructions to execute ONNX Runtime with the Standalone TVM (STVM) execution provider
title: TVM
description: Instructions to execute ONNX Runtime with the Apache TVM execution provider
parent: Execution Providers
nav_order: 8
---
# Standalone TVM (STVM) Execution Provider
# TVM (TVM) Execution Provider
{: .no_toc }
STVM is an execution provider for ONNX Runtime that is built on top of Apache TVM. It enables ONNX Runtime users to leverage Apache TVM model optimizations.
STVM EP is currently in "Preview". It's been tested to work on a handful of models on Linux, but not on Windows or MacOS.
TVM is an execution provider for ONNX Runtime that is built on top of Apache TVM. It enables ONNX Runtime users to leverage Apache TVM model optimizations.
The TVM EP is currently in "Preview". It's been tested to work on a limited set of ONNX models on Linux. Future enhancements will include support on Windows and MacOS.
__NOTE__: This TVM execution provider was developed as a _standalone_ TVM execution provider. The code repo has references to _stvm_ as the name for this EP.
## Contents
{: .no_toc }
@ -19,9 +21,9 @@ STVM EP is currently in "Preview". It's been tested to work on a handful of mode
## Build
To use the STVM EP in ONNX Runtime (ORT), users first need to build Apache TVM and ONNX Runtime.
To use the TVM EP in ONNX Runtime (ORT), users first need to build Apache TVM and ONNX Runtime.
Note: some python packages may need to be upgraded/downgraded because both TVM and ORT with the STVM EP use the Python API. Alternatively, use modify PYTHONPATH to solve these conflicts.
Note: some python packages may need to be upgraded/downgraded because both TVM and ORT with the TVM EP use the Python API. Alternatively, use modify PYTHONPATH to solve these conflicts.
### Build and configure TVM
@ -53,7 +55,7 @@ export PYTHONPATH=$TVM_HOME/python:${PYTHONPATH}
For more details on installing Apache TVM click [here](https://tvm.apache.org/docs/install/from_source.html)
### Build ONNX Runtime with the STVM Execution Provider
### Build ONNX Runtime with the TVM Execution Provider
In order to build ONNXRT you will need to have CMake 3.18 or higher. In Ubuntu 20.04 you can use the following commands to install the latest version of CMake:
@ -91,7 +93,7 @@ export PYTHONPATH=$ORT_PYTHON_HOME:${PYTHONPATH}
```
## Configuration options
STVM Executor Provider can be configured with the following provider options:
TVM Executor Provider can be configured with the following provider options:
```python
po = [dict(target=client_target,
target_host=client_target_host,
@ -100,7 +102,7 @@ po = [dict(target=client_target,
tuning_file_path=client_tuning_logfile,
input_names = input_names_str,
input_shapes = input_shapes_str)]
stvm_session = onnxruntime.InferenceSession(model_path, providers=["StvmExecutionProvider"], provider_options=po)
tvm_session = onnxruntime.InferenceSession(model_path, providers=["StvmExecutionProvider"], provider_options=po)
```
<br>
@ -117,7 +119,7 @@ input_shapes = "[1 3 224 224] [1 2]"
```
## Performance Tuning
TVM optimizes machine learning models through an automated tuning process that produces model variants specific to targeted hardware architectures. This process also generates 'tuning logs' that the STVM EP relies on to maximize model performance. These logs can be acquired for your model by either using TVM as described here:
TVM optimizes machine learning models through an automated tuning process that produces model variants specific to targeted hardware architectures. This process also generates 'tuning logs' that the TVM EP relies on to maximize model performance. These logs can be acquired for your model by either using TVM as described here:
AutoTVM:
https://tvm.apache.org/docs/how_to/tune_with_autotvm/index.html
@ -127,19 +129,19 @@ https://tvm.apache.org/docs/how_to/tune_with_autoscheduler/index.html
or by using logs generated through the OctoML platform (https://onnx.octoml.ai) using instructions [here](https://help.octoml.ai/en/articles/5814452-using-octoml-platform-logs-with-onnx-rt-tvm-ep)
Using the STVM EP with TVM tuning logs also requires users to turn off ONNX Runtime preprocessing. To do this, the following `SessionOptions()` can be used:
Using the TVM EP with TVM tuning logs also requires users to turn off ONNX Runtime preprocessing. To do this, the following `SessionOptions()` can be used:
```python
so = onnxruntime.SessionOptions()
so.graph_optimization_level = onnxruntime.GraphOptimizationLevel.ORT_DISABLE_ALL
stvm_session = onnxruntime.InferenceSession(model_path, sess_options=so, providers=["StvmExecutionProvider"], provider_options=po)
tvm_session = onnxruntime.InferenceSession(model_path, sess_options=so, providers=["StvmExecutionProvider"], provider_options=po)
```
## Samples
- [Sample notebook for ResNet50 inference with STVM EP](https://github.com/octoml/onnxruntime/blob/STVM_EP_PR/docs/python/inference/notebooks/onnxruntime-stvm-tutorial.ipynb)
- [Sample notebook for ResNet50 inference with TVM EP](https://github.com/octoml/onnxruntime/blob/STVM_EP_PR/docs/python/inference/notebooks/onnxruntime-stvm-tutorial.ipynb)
## Known issues
- At this moment, the STVM EP has only been verified on UNIX/Linux systems.
- At this moment, the TVM EP has only been verified on UNIX/Linux systems.
- CUDA/GPU support is still in pre-alpha mode and results are expected to change. It is recommended that only CPU targets are used.
- Some compatibility issues have been found between ONNX and Google protobuf. `AttributeError: module 'google.protobuf.internal.containers' has no attribute 'MutableMapping'`. This usually occurss during `import onnx` in any python scripts for protobuf version >= 3.19.0 and ONNX version <= 1.8.1. To resolve the issue Google protobuf and ONNX can be reinstalled separately or together using:
```bash

6
docs/execution-providers/index.md
View file

@ -29,10 +29,10 @@ ONNX Runtime supports many different execution providers today. Some of the EPs
---|---|---|---
|Default CPU|[NVIDIA CUDA](../execution-providers/CUDA-ExecutionProvider.md)|[Intel OpenVINO](../execution-providers/OpenVINO-ExecutionProvider.md)|
|[Intel DNNL](../execution-providers/oneDNN-ExecutionProvider.md)|[NVIDIA TensorRT](../execution-providers/TensorRT-ExecutionProvider.md)|[ARM Compute Library](../execution-providers/ACL-ExecutionProvider.md) (*preview*)|[Rockchip NPU](../execution-providers/RKNPU-ExecutionProvider.md) (*preview*)|
|[Standalone TVM](../execution-providers/STVM-ExecutionProvider.md) (*preview*)|[DirectML](../execution-providers/DirectML-ExecutionProvider.md)|[Android Neural Networks API](../execution-providers/NNAPI-ExecutionProvider.md)|[Xilinx Vitis-AI](../execution-providers/Vitis-AI-ExecutionProvider.md) (*preview*)|
|[TVM](../execution-providers/TVM-ExecutionProvider.md) (*preview*)|[DirectML](../execution-providers/DirectML-ExecutionProvider.md)|[Android Neural Networks API](../execution-providers/NNAPI-ExecutionProvider.md)|[Xilinx Vitis-AI](../execution-providers/Vitis-AI-ExecutionProvider.md) (*preview*)|
||[AMD MIGraphX](../execution-providers/MIGraphX-ExecutionProvider.md) (*preview*)|[ARM-NN](../execution-providers/ArmNN-ExecutionProvider.md) (*preview*)|
||[Standalone TVM](../execution-providers/STVM-ExecutionProvider.md) (*preview*)|[CoreML](../execution-providers/CoreML-ExecutionProvider.md) (*preview*)|
|||[Standalone TVM](../execution-providers/STVM-ExecutionProvider.md) (*preview*)|
||[TVM](../execution-providers/TVM-ExecutionProvider.md) (*preview*)|[CoreML](../execution-providers/CoreML-ExecutionProvider.md) (*preview*)|
|||[TVM](../execution-providers/TVM-ExecutionProvider.md) (*preview*)|
### Add an Execution Provider

4
index.html
View file

@ -267,8 +267,8 @@
<span>ArmNN (Preview)</span></div>
<div class="col-lg-2dot5 col-md-3 r-option version" role="option" tabindex="-1" aria-selected="false" id="MIGraphX">
<span>MIGraphX (Preview)</span></div>
<div class="col-lg-2dot5 col-md-3 r-option version" role="option" tabindex="-1" aria-selected="false" id="STVM">
<span><abbr>STVM (Preview)</abbr></span></div>
<div class="col-lg-2dot5 col-md-3 r-option version" role="option" tabindex="-1" aria-selected="false" id="TVM">
<span><abbr>TVM (Preview)</abbr></span></div>
<div class="col-lg-2dot5 col-md-3 r-option version" role="option" tabindex="-1" aria-selected="false" id="RockchipNPU">
<span>Rockchip NPU (Preview)</span></div>
<div class="col-lg-2dot5 col-md-3 r-option version" role="option" tabindex="-1" aria-selected="false" id="VitisAI">

10
js/script.js
View file

@ -760,19 +760,19 @@ var validCombos = {
"linux,Python,X64,DNNL":
"Follow build instructions from&nbsp;<a href='https://aka.ms/build-ort-mkldnn' target='_blank'>here</a>",
"linux,Python,X64,STVM":
"linux,Python,X64,TVM":
"Follow build instructions from&nbsp;<a href='https://aka.ms/build-ort-stvm' target='_blank'>here</a>",
"linux,Python,X86,STVM":
"linux,Python,X86,TVM":
"Follow build instructions from&nbsp;<a href='https://aka.ms/build-ort-stvm' target='_blank'>here</a>",
"linux,Python,ARM32,STVM":
"linux,Python,ARM32,TVM":
"Follow build instructions from&nbsp;<a href='https://aka.ms/build-ort-stvm' target='_blank'>here</a>",
"linux,Python,ARM64,STVM":
"linux,Python,ARM64,TVM":
"Follow build instructions from&nbsp;<a href='https://aka.ms/build-ort-stvm' target='_blank'>here</a>",
"linux,Python,AMD,STVM":
"linux,Python,AMD,TVM":
"Follow build instructions from&nbsp;<a href='https://aka.ms/build-ort-stvm' target='_blank'>here</a>",
"linux,C-API,X64,OpenVINO":