From fc16c5cc0db14f4d6b21eaf89f19217bb4942631 Mon Sep 17 00:00:00 2001 From: Preetha Veeramalai Date: Tue, 12 Jul 2022 11:26:12 +0530 Subject: [PATCH] Ov ep docx update 1.12 (#12121) * updating docs for torch ort inference * Add provider option for enable_dynamic_shapes Co-authored-by: saipj --- docs/execution-providers/OpenVINO-ExecutionProvider.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/docs/execution-providers/OpenVINO-ExecutionProvider.md b/docs/execution-providers/OpenVINO-ExecutionProvider.md index c64cf24f0d..82c98e0fd6 100644 --- a/docs/execution-providers/OpenVINO-ExecutionProvider.md +++ b/docs/execution-providers/OpenVINO-ExecutionProvider.md @@ -26,7 +26,7 @@ Pre-built packages and Docker images are published for OpenVINO™ Execution Pro ## Requirements -|ONNX Runtime|OOpenVINO™|Notes| +|ONNX Runtime|OpenVINO™|Notes| |---|---|---| |1.11.0|2022.1|[Details](https://github.com/intel/onnxruntime/releases/tag/v4.0)| |1.10.0|2021.4.2|[Details](https://github.com/intel/onnxruntime/releases/tag/v3.4)| @@ -289,6 +289,7 @@ The following table lists all the available configuration options and the Key-Va | blob_dump_path | string | Any valid string path on the hardware target | string | Explicitly specify the path where you would like to dump and load the blobs for the save/load blob feature when use_compiled_network setting is enabled . This overrides the default path.| | context | string | OpenCL Context | void* | This option is only alvailable when OpenVINO EP is built with OpenCL flags enabled. It takes in the remote context i.e the cl_context address as a void pointer.| | enable_opencl_throttling | string | True/False | boolean | This option enables OpenCL queue throttling for GPU devices (reduces CPU utilization when using GPU). | +| enable_dynamic_shapes | string | True/False | boolean | This option if enabled works for dynamic shaped models whose shape will be set dynamically based on the infer input image/data shape at run time in CPU. This gives best result for running multiple inferences with varied shaped images/data. | Valid Hetero or Multi or Auto Device combinations: HETERO:,,... @@ -568,3 +569,8 @@ In order to showcase what you can do with the OpenVINO™ Execution Provider for ### Tutorial on how to use OpenVINO™ Execution Provider for ONNX Runtime python wheel packages [Python Pip Wheel Packages](https://www.intel.com/content/www/us/en/artificial-intelligence/posts/openvino-execution-provider-for-onnx-runtime.html) + +## Accelerate inference for PyTorch models with OpenVINO Execution Provider (Preview) + +ONNX Runtime for PyTorch is now extended to support PyTorch model inference using ONNX Runtime. +It is available via the torch-ort-inference python package. This preview package enables OpenVINO™ Execution Provider for ONNX Runtime by default for accelerating inference on various Intel® CPUs, Intel® integrated GPUs, and Intel® Movidius™ Vision Processing Units - referred to as VPU. For more details, see [torch-ort-inference](https://github.com/pytorch/ort#accelerate-inference-for-pytorch-models-with-onnx-runtime-preview).