mirror of
https://github.com/saymrwulf/onnxruntime.git
synced 2026-05-18 21:21:17 +00:00
65 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Thiago Crepaldi
|
8a890ddfd7
|
Sync ORTModule branch with master and fix tests (#6526)
* Deprecate Python global configuration functions [Part 1] (#5923) Enable options to be set via execution provider (EP)-specific options and log deprecation warning from current global configuration functions. * remove dnnl_dll_path from post build copy (#6142) * Model Fusion For Bart (#6105) Fusion fix for Bart models * Unify IExecutionProvider and IExecutionProviderFactory interfaces (#6108) * Remove Provider_IExecutionProvider and make the internal IExecutionProvider usable by shared providers * Change Provider_IExecutionProviderFactory to be the core version. * Enable running the mnist_training sample without cuda (#6085) Signed-off-by: George Nash <george.nash@intel.com> * nnapi add min max support (#6117) * Fix CUDA test hang: (#6138) - Make condition check in `CUDAAllocatorTest` to ensure CUDA device is present. * Fix TensorRT kernel conflict issue for subgraphs of control flow operators (#6115) * add static subgraph kernel index * change kernel naming to avoid conflicts * Add gradient registration for Abs. (#6139) * Partition initial optimizer state for Zero-1 (#6093) * Initial changes * Working changes * Working changes * Cleanup * fix windows CI * Review comments * review comments * Fix edge case in BFCArena where allocation failures could lead to an infinite loop. (#6145) #4656 * Revert "work around of the build break in mac (#6069)" (#6150) This reverts commit |
||
|
sfatimar
|
8168c91978
|
Sahar/fix documentation shared lib (#5926)
* Update OpenVINO-ExecutionProvider.Md update openvino-executionprovider.md for shared library * Update Build.md updated --build_shared_lib flag for building openvino shared provider lib * Update Dockerfile.openvino building for shared library with the new changes for openvino shared lib * Revert "Update Build.md" This reverts commit c9cf5fee76be7fdc10cadf07259f1d4ed5b45b93. * Revert "Update Dockerfile.openvino " This reverts commit e1624e4f93a4cfb425b6f21d7fb71b299a146740. * Update OpenVINO-ExecutionProvider.md fix documentation to the shared library Co-authored-by: sfatimar <sahar.fatima@intel/com> |
||
|
stevenlix
|
1068f3eb87
|
Use flatbuffers for INT8 calibration table (de)serialization in TensorRT EP (#5873)
* add int8 * support both native TRT cal table and ORT cal table * add more comments * Update env variable name and check platform availability for int8/fp16 * add backward compatibility on old env var ORT_TENSORRT_ENGINE_CACHE_PATH and switch to flatbuffers for ort cal table deserialization |
||
|
stevenlix
|
dfea92925c
|
Add calibration based INT8 quantization to TensorRT EP (#5842)
* add int8 * support both native TRT cal table and ORT cal table * add more comments * Update env variable name and check platform availability for int8/fp16 |
||
|
S. Manohar Karlapalem
|
ff58f621fa
|
Remove nGraph Execution Provider (#5858)
* Remove nGraph Execution Provider Pursuant to nGraph deprecation notice: https://github.com/microsoft/onnxruntime/blob/master/docs/execution_providers/nGraph-ExecutionProvider.md#deprecation-notice **Deprecation Notice** | | | | --- | --- | | Deprecation Begins | June 1, 2020 | | Removal Date | December 1, 2020 | Starting with the OpenVINO™ toolkit 2020.2 release, all of the features previously available through nGraph have been merged into the OpenVINO™ toolkit. As a result, all the features previously available through ONNX RT Execution Provider for nGraph have been merged with ONNX RT Execution Provider for OpenVINO™ toolkit. Therefore, ONNX RT Execution Provider for **nGraph** will be deprecated starting June 1, 2020 and will be completely removed on December 1, 2020. Users are recommended to migrate to the ONNX RT Execution Provider for OpenVINO™ toolkit as the unified solution for all AI inferencing on Intel® hardware. * Remove nGraph Licence info from ThirdPartyNotices.txt * Use simple Test.Run() for tests without EP exclusions To be consistent with rest of test code. * Remove nGraph EP functions from Java code |
||
|
Justin Stoecker
|
bd236ecc26
|
Switch to unified DirectML 1.4.0 redistributable (#5794)
Transitions from the ORT-only DML NuGet (hosted on the onnxruntime_public feed) to the new unified DirectML NuGet (Microsoft.AI.DirectML) on nuget.org. In addition, the Microsoft.AI.MachineLearning (WinML) and Microsoft.ML.OnnxRuntime.DirectML packages now take a dependency on the Microsoft.AI.DirectML package. This means we can remove the extra copy of DML binaries in these packages since they will be installed by the DML package. |
||
|
stevenlix
|
54de618c2e
|
Improve TensorRT engine caching (#5737)
* add profile caching to improve engine caching feature * Add comments * fix typo * add decryption for engine caching * Update tensorrt_execution_provider.cc * Update tensorrt_execution_provider.cc * Update tensorrt_execution_provider.cc * Update tensorrt_execution_provider.cc * Update tensorrt_execution_provider.cc * update onnx-tensorrt submodule * set opt profile to max value of the range * add hash to engine/profile name * Add calibration based INT8 quantization * add an option to enable both FP16 and INT8 * Update tensorrt_execution_provider.cc * add env variable to specify calibration file name * clean up code * Add comments and update TRT document * enable tensorrt basic test and add EngineCachingTest * clean up * update envrionment variable in the test * clean up |
||
|
Maajid khan
|
a84a058f9e
|
[OpenVINO-EP] Enabling Multi Device support (#5740)
* Enabling Multi Device support for UEP Signed-off-by: MaajidKhan <n.maajidkhan@gmail.com> * Minor fix added *Added a simple fix to determine OpenVINO version for Arm build as well Signed-off-by: MaajidKhan <n.maajidkhan@gmail.com> |
||
|
Johannes Bannhofer
|
6f6dd0b869
|
added missing flag ORT_TENSORRT_DUMP_SUBGRAPHS (#5724)
[DOCUMENTATION] added descriptionof the function ORT_TENSORRT_DUMP_SUBGRAPHS to the documentation |
||
|
Maajid khan
|
d98062da0c
|
[OpenVINO-EP] Hetero support (#5627)
* Implement Hetero in UEP * Added security checks to take valid Hetero combinations as device type * Integrating Hetero features * Get the statistics Report in Debug Mode Signed-off-by: MaajidKhan <n.maajidkhan@gmail.com> * Passing right device type for vadm_baackend Added simple fix to pick the right device type when using vadm_backend with Hetero as well. Signed-off-by: MaajidKhan <n.maajidkhan@gmail.com> * Fixed batching logic for 2020.4 and above * Fixed flake8 PEP8 errors Signed-off-by: MaajidKhan <n.maajidkhan@gmail.com> * Minor Fixes Added *Added security checks for device_type passed in for Hetero build during run time *code cleanup Signed-off-by: MaajidKhan <n.maajidkhan@gmail.com> * Minor changes Added *Fixed batch_size bug in vadm_backend *code cleanup *Documentation updated for Hetero Signed-off-by: MaajidKhan <n.maajidkhan@gmail.com> Co-authored-by: suryasidd <surya.siddharth.pemmaraju@intel.com> |
||
|
Maajid khan
|
ddf83d1ace
|
Maajid/multi threading 2 (#5568)
* Enabled multi-threading for OpenVino EP ->Enabled support for concurrent_session_runs *Run UEP using concurrent_session_runs > 1 *Enabled support for ORT_PARALLEL ExecutionMode ->Documentation Added for Enabling MultiThreading Signed-off-by: MaajidKhan <n.maajidkhan@gmail.com> * Minor Fixes added *Configure the value of nireq during Runtime *Documentation typos rectified and details added for Multi_Threaded Inference Signed-off-by: MaajidKhan <n.maajidkhan@gmail.com> * Some checks added for this fix *Added checks to invalidate wrong nireq value and assigned it to default value of 8 *Added new config options for enable_vpu_fast_compile which were changed w.r.t OpenVINO_2021.1 Release Signed-off-by: MaajidKhan <n.maajidkhan@gmail.com> |
||
|
Olivia Jain
|
1e4b259d28
|
Updating EP docs with Onnxruntime API calls (#5503)
* updating examples with current api calls * Fixing capitalization in api calls, adding RKNPU update * Correcting nuphar and rknpu ep api calls * Include creating session in readme |
||
|
sfatimar
|
6d2a30eae3
|
[OPENVINO-EP] 2021.1 Release (#5431)
* Cmake changes for 2021.1 * added new ov version 2020.1 for faster rcnn * Added missing defs * equal op modified * changes to incoroporate faster rcnn * backend util.cc * hddl_plugin_config.hpp is depreceated . instead use hddl_config.hpp * changing myriad precision bool to i32 * gather is not enabled for gpu * conv2D and pooltest auto_pad attribute should not be null * negative indices are not valid for scatter op in myriad * non max suppression op only supported in faster rcnn mode * maxpool indices output is not supported * Cleaned redundant code in backends * Added ifdefs for HDDL config * cast output dimensions check topk operator k input it seems only resolved for myriad as it is throwing issues for ask rcnn . need to verify * we are limiting the subgraph size to 3 here * taking care of review comments * Fixed minor bugs * Modified Slice op checks * Added NonZero, Upsample * Removed TopK if it's in the middle of a subgraph * incorporated upsample conditions too * Dockerfile changes for 2021.1 release * dockerfile aptkey update * Minor fixes * ceil condition added again * Fixed few gpu models * Disabled LSTM and yolov3 in ModelTests * python softmax cross entropy tests and negative log likelihood * Update Build.md Updated for openvino 2021.1 * Update OpenVINO-ExecutionProvider.md update openvino execution provider for 2021.1 * Update READMe.md updated new openvino version * Update Dockerfile.openvino added environment variable for DEBIAN Frontend * Fixed myriad models * Fixed gather condition * Fixed mask rcnn model on myriad * Modified Gather condition * set default target of MCR dockerfile to MYRIAD_FP16 * Fixed tinyolov3 on CPU * Update OpenVINO-ExecutionProvider.md update openvino execution provider documentation * Update Dockerfile.openvino Removed environment variable * Update OpenVINO-ExecutionProvider.md update image manipulation networks supported * Update onnx_backend_test_series_filters.jsonc removed test_upsample_nearest from cpu test cases * New InternalCI changes for 2021.1 * Full protobuf removed for OpenVINO * Protobuf added * Updated with apt installation for openvino * Revert the testing changes * Reverted testing changes * File permessions are changed to original * Deleted openvino installation and cmake change * Optimized Dockerfile Removed unnecessary cmake installation, numpy * Added missing ifdefs * delete array fix * backend_utils.cc output_shape * Revert "set default target of MCR dockerfile to MYRIAD_FP16" This reverts commit 928d3e2b71e2f589cf51dacd3a133951cf9ca18d. Co-authored-by: suryasidd <surya.siddharth.pemmaraju@intel.com> Co-authored-by: sfatimar <sahar.fatima@intel/com> Co-authored-by: suryasidd <48925384+suryasidd@users.noreply.github.com> Co-authored-by: S. Manohar Karlapalem <manohar.karlapalem@intel.com> Co-authored-by: Aravind <aravindx.gunda@intel.com> Co-authored-by: Aravind Gunda <38353114+gundaarx@users.noreply.github.com> |
||
|
manashgoswami
|
b5caa7cb12
|
Updated docs: Execution Provider overview (#5328)
* Update ReleaseManagement.md * Create ONNX_Runtime_Execution_Providers.md * Create ONNX_Runtime_EP3.png * Create ONNX_Runtime_EP2.png * Create ONNX_Runtime_EP1.png * Delete ONNX_Runtime_Execution_Providers.md * Create README.md * Update README.md * commit * Updated in error. Revert "Update ReleaseManagement.md" This reverts commit 8530bd5fd46aebce3a6d6055d8952ae4f6458c4e. * Create ONNX_Runtime_Execution_Providers.md * Create ONNX_Runtime_EP3.png * Create ONNX_Runtime_EP2.png * Create ONNX_Runtime_EP1.png * Delete ONNX_Runtime_Execution_Providers.md * Create README.md * Update README.md * commit * Updated in error. Revert "Update ReleaseManagement.md" This reverts commit 8530bd5fd46aebce3a6d6055d8952ae4f6458c4e. * Update ReleaseManagement.md * Update .gitignore * Update README.md * Update README.md |
||
|
Dwayne Robinson
|
6ad39819c2
|
Update DirectML Nuget to 1.3.0 (#5274)
Update to 1.3.0 |
||
|
George Wu
|
3147bc00c3
|
update TensorRT docs (#5238)
* doc updates TensorRT * update * update * fix warning * newline * format |
||
|
KeDengMS
|
ce3b67e0cd
|
[Python] Move symbolic_shape_infer from nuphar to tools (#5162)
* [Python] Move symbolic shape inference from nuphar to tools * Fix PEP8 ERROR |
||
|
S. Manohar Karlapalem
|
584638e5d3
|
Corrects doc typos and formatting (#5201) | ||
|
S. Manohar Karlapalem
|
f7edf0aa57
|
[OpenVINO-EP] Enable EP config options for VPU hardware (#5119)
* Added config flags for VPU Fast Recompile * clean-up ifdefs * Add VPU Fast compile config option Adds an option that enables Fast compilation of models to VPU hardware specific format. * Add config option to choose specific device id for inference Inference of all subgraphs will be scheduled only on this device even if other devices of the same type are available. * Add Python API to list available device IDs * code cleanup * Add second C/C++ API with settings string parameter Adds an additional C/C++ API that allows passing multiple key-value pairs for settings as a single string. Multiple settings are delimited by '\n' while the key and value within a setting are delimited by '|'. * Append 'Ex' to the extended C/C++ API * Use set_providers Py API to set config options. Uses Session.set_providers Python API to set EP runtime config options as key/val pairs Deprecated older module function definitions for config settings. Updates documentation. * avoid globals for py config options where possible Co-authored-by: intel <you@example.com> |
||
|
Ashwini Khade
|
cd56ab197c
|
csharp build documentation (#5121) | ||
|
suryasidd
|
3a00b50cf8
|
[OpenVINO-EP] Updating OpenVINO EP to 2020.4 (#4836)
* Removed building ngraph from source * Disabled some tests temporarily * Enabled softmax for all dims * Added onnx importer to link libraries * int64 changes * fixed * temp * slice update start and end need to be initializer * Disabled GatherND, ScatterND, ReverseSequence operators * Added supported ops instead of unsupported ops * Set precision only for CPU * Removed some unecessary conditions * Fixed segfault in slice * Softmax restriction removed * changes * Setting precision for all plugins * Changes added to include precision and supported ops for gpu and vpu * branch op support * checking for disabled python test failure * mapped input names and tensors directly rather than copying which was leading to mismatch * last index is not supported mkldnn does not support pow between integers * included the code changes * Rename inner-scoped variable to avoid MSVC warning * applied changed to vadm as well and removed the utility function getinputtensors() completely * OpenVINO multi version support: CMake changes * OpenVINO multi version support: C++ support * removed commented code * Remove redundant code lines * Revert "Rename inner-scoped variable to avoid MSVC warning" This reverts commit 2f650493162675bc6fb70730de9656ec400be332. Merged separately in master. * vadm changes disabled reduction op test * putting test_gather_negative_indices in unsupported list for now * Update MCR Dockerfile with 2020.4 Installs OpenVINO 2020.4 from deb packages via APT tool. * Update build docs with 2020.4 info * Update dockerfile with OV 2020.4 info Instructions for building OpenVINO based docker image no longer require downloading installer package as it is installed by the dockerfile using OpenVINO 2020.4 APT package for Ubuntu 18.04 * Added constant folding bypass logic * Added cout statements for ci * Added NDEBUG flag for debug symbols * Update Ops info in docs * fixes multiple unit tests * mathoptest.ceil disabled for gpu and myriad * activation test temp disabled * Fix models for CPU * Fixed a syntax error * local cmmit * fixing unit tests for myriad * Fixed Variadic Split, Topk issues * fix_model commit * Fix models in myriad * Added ifdefs for OpenVINO 2020.4 * temp * made some changes to not operator * Added unused parameter * relu enabled * Fixed bug in Conv output * Consolidated GPU failing tests into one category * Made it compatible to InternalCI 2020.4 * Made changes for ngraph * Disabled test for mask,fastercnn,tinyyolov3 * Removed proxy for ci * run_dockerbuild.sh restored to same version * run_dockerbuild.sh restored to same version * run_dockerbuild.sh restored to same version * Updated documentation for 2020.4 * Removed FP32 to FP16 transformation for GPU * Disabled Coreml-FNS-Candy model test * Added FP16 transformations Co-authored-by: sfatimar <sahar.fatima@intel.com> Co-authored-by: Manohar Karlapalem <manohar.karlapalem@intel.com> Co-authored-by: sfatimar <sahar.fatima@intel/com> Co-authored-by: sfatimar <64512376+sfatimar@users.noreply.github.com> Co-authored-by: intel <you@example.com> Co-authored-by: gundaarx <aravindx.gunda@intel.com> |
||
|
Hariharan Seshadri
|
c878ecbbe0
|
Sahar/csharp support openvino (refined) (#4835)
* Sahar/csharp support openvino (#4703) * Temp changes and include openvino to ensure nuget package is created with linux till we configure azure ci pipeline * string id change * native nuget indentation changes * documentation changes * Update Openvino_execution_provider.md Documentation includes openvino execution provider * Update OpenVino-ExecutionProvider.md update details to build csharp api for openvino execution provider . * vadm backend revert * Update Openvino-Execution-Provider.md updated for review comments * Update OpenVino-Execution-Provider.md * Update OpenVINO-ExecutionProvider.md * nuget package custome support for openvino change in native nuget spec python script for including linux runtime * change to make path to boolean flag * removed the tab * Update OpenVINO-ExecutionProvider.md updated for review comments * chnages to include pep8 warnings modification to documentation Co-authored-by: saharfraza <sfatima.3001@gmail.com> Co-authored-by: sfatimar <sahar.fatima@intel/com> * Changes to include csharp support for openvino * Fix flake error * Fix Co-authored-by: sfatimar <64512376+sfatimar@users.noreply.github.com> Co-authored-by: saharfraza <sfatima.3001@gmail.com> Co-authored-by: sfatimar <sahar.fatima@intel/com> |
||
|
George Wu
|
94a6f50af6 |
Revert "Sahar/csharp support openvino (#4703)"
This reverts commit
|
||
|
sfatimar
|
0a0ac70eec
|
Sahar/csharp support openvino (#4703)
* Temp changes and include openvino to ensure nuget package is created with linux till we configure azure ci pipeline * string id change * native nuget indentation changes * documentation changes * Update Openvino_execution_provider.md Documentation includes openvino execution provider * Update OpenVino-ExecutionProvider.md update details to build csharp api for openvino execution provider . * vadm backend revert * Update Openvino-Execution-Provider.md updated for review comments * Update OpenVino-Execution-Provider.md * Update OpenVINO-ExecutionProvider.md * nuget package custome support for openvino change in native nuget spec python script for including linux runtime * change to make path to boolean flag * removed the tab * Update OpenVINO-ExecutionProvider.md updated for review comments * chnages to include pep8 warnings modification to documentation Co-authored-by: saharfraza <sfatima.3001@gmail.com> Co-authored-by: sfatimar <sahar.fatima@intel/com> |
||
|
stevenlix
|
77c69a0325
|
Upgrade TensorRT to v7.1.3.4 (#4704)
* upgrade to TensorRT 7.1.3.4 * Upgrade onnx-tensorrt parser for TensorRT 7.1.3.4 * fix format issue * fix format issue * fix format issue * Update tensorrt_execution_provider.cc * change cmake version to 3.14 * Remove --msvc_toolset 14.16 * change to onnxruntime::make_unique * use onnxruntime::make_unique * disable some tests for TensorRT * disable some tests for TensorRT * Update upsample_op_test.cc * Update tile_op_test.cc * disable some tests for TensorRT * Update constant_of_shape_test.cc * update parser * Update Dockerfile.ubuntu_tensorrt |
||
|
gwang-msft
|
c2ec3b734b
|
[Android NNAPI EP] Remove dependency on external JD/DNNLibrary (#4576)
* remove dependency of external jd-dnnlibrary * remove extra variables not used any more * update /cgmanifest.json |
||
|
stevenlix
|
0ebe2fab51
|
Refactor TensorRT EP code to better handle dynamic shape subgraphs (#4504)
* build engine in runtime for dynamic shape subgraphs * Update TensorRT-ExecutionProvider.md * Update TensorRT-ExecutionProvider.md * fix build issue * Add more instructions on how to use engine caching * add precision to trt node name * Update tensorrt_execution_provider.cc * Update tensorrt_execution_provider.cc |
||
|
S. Manohar Karlapalem
|
ceedf126a2
|
[nGraph] Deprecation notice for nGraph EP (#4344) | ||
|
Shucai Xiao
|
bfc888613f
|
Migraphx improvements (#4328)
* Add amd migraphx execution provider to onnx runtime * rename MiGraphX to MIGraphX * add migraphx EP to tests * support multiple program output * disable more tests * backup changes related to program multiple outputs * remove logging code * remove unnecessary changes in migraphx_execution_provider.cc * add migraphx EP to tests * add input requests of the batchnorm operator * add to support an onnx operator PRelu * update migrapx dockerfile and removed one unused line * chagnes related to support dynamic input shape * fix build error * code backup * code backup * version that has 106 models run correctly * code backup * code backup * remove unnecessary print info * code backup * code backup * code backup * code backup * code backup * code backup * changes corresponding to migraphx change * fix merge conflict * minor code cleanup * code cleanup * remove unnecessary code * remove unnecessary code * add to support more constant folding analysis * more constant folding checking for shape input * add env var to control whether fp16 is enabled. Modify docker file to use ROCM3.3 * fix function name to avoid build error * add build and execution instruction for migraphx execution provider * added more build instructions * fixed a small format error * a minor change * fix review comments * another minor change * additional refinement of the documents * additional changes * remove unnecessary changes in the dockfile * additional changes for the dockerfile * code change backup * fix errors related to a few unit tests * fix a build error related to api change * fix unit test errors by either disabling the test or fix related isssues * remove unnecessary log info * sync submodule tvm with master * remove unnecessary changes * remove an unnecessary code line * refine documents for addition example |
||
|
jornt-xilinx
|
c55f6d76be
|
[Vitis-AI EP] Fix to enable multi-output subgraphs inside Vitis-AI EP + edit docs (#4171) | ||
|
Andrews548
|
62b44527e5
|
Add ArmNN Execution Provider (#3714)
* Add ArmNN Execution Provider Add a new execution provider targeting Arm architecture based on ArmNN. Validated on NXP i.MX8QM CPU with ResNet50, MobileNetv2 and VGG models. reviewed-by: mike.caraman@nxp.com * Minor fixes - renamed onnxruntime_ARMNN_RELU_USECPU to onnxruntime_ARMNN_RELU_USE_CPU - fixed acl typo * remove extra includes. added exception for ArmNN in test * fix indentation * Separated the activation implementation from the cpu and fixed the blockage from the endif Co-authored-by: Andrei-Alexandru <andrei-alexandru.avram@nxp.com> |
||
|
Dwayne Robinson
|
51d78bc5e6
|
Fix DML EP doc link to C API (#4105)
Path used "\" instead of "/". |
||
|
edelaye
|
64b5f7edf6
|
Initial release of Vitis-AI Execution Provider (#3771)
* Initial release of Vitis-AI Execution Provider * Add documentation, fix for onnxruntime::Model changes and use stringstream instead of file dump for model passing * - Add Vitis-AI docker file - Add online quantization flow Vitis-AI execution provider - Fix remarks * - Add fatal error build message for Vitis-AI cmake build on Windows - Fix pep8 issue in build.py - Add Vitis-AI execution provider example in docs Co-authored-by: Elliott Delaye <elliott@xilinx.com> Co-authored-by: Jorn Tuyls <jornt@xilinx.com> Co-authored-by: Jorn Tuyls <jtuyls@users.noreply.github.com> |
||
|
Faith Xu
|
b8a255e1b5
|
Doc Updates for Build (#3976)
* Initial update of readme * Readme updates * Review of consolidated README (#3930) * Proposed updates for readme (#3953) I found some of the information was duplicated within the doc, so attempted to streamline * Fix links * More updates - fix build instructions - nodejs doc reorganization - roadmap update - version fixes * Update ORT Server build instructions * More doc cleanup * fix python dev notes name * Update nodejs and some links * sync eigen version back to master * Minor fixes * add nodsjs to sample table of content * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * address PR feedback * address PR feedback * nodejs build instruction * Update Java instructions to include gradle * Roadmap refresh Reformat some data, fix link, minor rewording * Clarify Visual C++ runtime req Co-authored-by: Nat Kershaw (MSFT) <nakersha@microsoft.com> Co-authored-by: Prasanth Pulavarthi <prasantp@microsoft.com> Co-authored-by: manashgoswami <magoswam@microsoft.com> |
||
|
Jeff Bloomfield
|
e6da5946d1
|
Update DML Nuget version and DML EP Doc (#3945)
Update DML Nuget version and DML EP Doc |
||
|
airockchip
|
edaf8a542c
|
Initial PR for RKNPU execution provider (#3609)
* Initial RKNPU execution provider
* Init
* Support Ops:
Conv, Relu, Clip, LeakyRelu,
MaxPool, AveragePool, GlobalAveragePool,
Concat, Softmax, BatchNormalization, Gemm,
Add, Mul, Sub,
Reshape, Squeeze, Unsqueeze,
Flatten, Transpose,
QLinearConv, DequantizeLinear
* Add rknpu unittest
* Update BUILD.md and Add RKNPU-ExecutionProvider.md
* misc code update
* fix CLIP accuracy issue.
* fix "Error: Duplicate definition of name".
* move rknpu_ddk out of onnxruntime submodule.
* remove temporary code.
* add rknpu namespace.
* update misc of node_attr_helper
* add const & comment for onnx_converter
* add const & comment for shaper
* unify variable name
Co-authored-by: dkm <dkm@rock-chips.com>
Co-authored-by: George Wu <jywu@microsoft.com>
|
||
|
Jeff Bloomfield
|
d5b2cd7493
|
Add performance best practices to DML EP doc (#2859)
* Add performance best practices to DML EP doc Co-authored-by: Jeff <38966965+jeffbloo@users.noreply.github.com> |
||
|
suryasidd
|
e529464a12
|
Limit the number of models run on OpenVINO (#3742)
* Removed NMS from supported list |
||
|
S. Manohar Karlapalem
|
6d4f2f5bf9
|
OpenVINO EP v2.0 (#3585)
* Added FP16 transformations
* Revert "Added CMAKE_BUILD_TYPE to make building dynamic"
This reverts commit d3e17af1af655cfdc4d2fec33f52055caa525e85.
* Added FP16 transformations for FP16 builds
* Backend logic cleanup
Cleans the backend(intel_graph.*) code in the following ways:-
1. Minimize global usage: Since all the IR graphs need to be
re-generated on every Infer, it is bad practice to rely on globals
for their saving and usage as there would be multiple readers and
writers to the same global variable leading to incorrect usages or
contentions. This change replaces globals with locals where possible.
This change also fixes an existing bug with due to
incorrect global usage.
2. Remove all unused functions.
3. Remove all unused headers and prepocessor directives.
* removed commented out code
* Disabled default optimization for Intel EP
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Fix missed plugins.xml for python bindings
* Fixed the build after latest master changes
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Disabled unsupported ops for accelerators
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Added some more disabled ops
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Added environment variable to enable debugging
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Added more debug statements
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Fixed unsupported ops list for GPU and VPU
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Fixed unsqueeze unit tests
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Added error message to the status
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Overwrite Model proto with shape info from data
Overwrites the shape info of Model proto with the shape from
actual input data. Needed for inferring models with Dynamic
shapes.
* Removed print statement and disabled where op
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Disabled Reshape with Empty initializer
* Added more debug statements for 1P
* Don't allow 1D inputs with symbol for dimension
* Disabled some 3rd phase ops
* Disabled split and added zero dimension check for OutputDefs
* Cleanup zero dimensionality check
* Added different data type check for inputs and initializers
* Added conditions for Mod, Cast and Pad
* Removed unused variable
* Disabled scan and added conditions for squeeze
* Added changes for fixing all C++ unit tests
* Implements Backend Manager class for caching
Backend Manager provides a layer of indirection between EP interface
and OV backend that provides caching services for models with
symbolic dims in input shapes.
* clean up commented blocks
* clang-formatting
* Read I/O type info from ModleProto
Read the tensor element type information from ModelProto object,
as FusedNode is no longer available.
* code cleanup
* clang-formatting
* Added print statement for jenkins
* Disabled some python tests
* Changed the path of convert fp32 to fp16 hpp
* Added conditions for BatchNorm in GetCapability
* Fixed failed tests
* Revert "Added conditions for BatchNorm in GetCapability"
This reverts commit c3c28c3b00d27892c42546b35dacdd807a48ee90.
* Added Intel to onnxruntime backends
* pick up vars set by OV package setupvars.sh
* Added conditions for Identity
* remove a few cout prints
* Added conditions for GPU_FP32 unit tests
* Revert "pick up vars set by OV package setupvars.sh"
This reverts commit 8199e029c03eae21a1a7ef6bfdc93d00e5d0198b.
* Commented out fatal message for protobuf
* Might need to be removed
* Add interface class for current backend
* moved common logic to base class
* simplified cpu backend
* Removed unused headers
* use vectors to save i/o tensors for windows compatibility
* move utils fxns to backend_utils namespace
* rename ov_backend to ibackend
* Factory pattern for backend creation
* rename CPU backend to Basic backend
* renamed to vad-M and added to factory list
* Added conditions for VPU
* Added print statements
* Changed the logic for checking for symbolic shapes
* Modified logic for zero dimension check
* Removed VPU single dimension condition
* Removed comments
* Modified logic in DimensionCheck method
* Remove legacy OpenVINO EP
Remove all the legacy code for OpenVINO EP. UEP code will take its
place going forward.
This change does NOT remove OVEP files in the following areas asa
they will be reused by UEP:-
1. Documentation: All .md files
2. Docker releated files
3. Python bindings
4. Java bindings
5. C# bindings
6. ORT Server
7. CI pipeline setup files
* Rename Intel EP to OpenVINO EP
* Added unique names to the subgraphs
* Removed subgraphs with only constant inputs
* Modified subgraph partitioning algorithm to remove const input subgraphs
* Apply suggestion to onnxruntime/core/providers/openvino/openvino_execution_provider.cc
* Tracking output names to fix the output order bug
* Changed output names to a unordered map
* Modified logic to check for symbolic input shapes
* Fixed a bug in Reshape check
* Added empty model path to Model constructor
* Made necessary changes to cmake to build from the binary package
* Changed INTEL_CVSDK_DIR to INTEL_OPENVINO_DIR
* Enable dyn device selection with C++ API
* Added Round operator to unsupported list
* Modified subgraph partition logic for MYRIAD
* Removed supported ops from the list
* Enable dyn dev selection in Py API's
* Add documentation for dynamic device selection
* Use MYRIAD || HDDL instead of VPU
* Removed temporary cast of Int64 to FP32
* Disabled unit Tests for CPU_FP32 and GPU_FP32
* Removed default "CPU" from unit tests to allow overriding
* Removed ops Concat, Squeeze, Unsqueeze from unsupported list
* Get the device id from info
* Removed overwriting device_id and precision
* Enabled ConvTranspose and EyeLike
* Reordered unsupported ops in alphabetical order
* Fixed syntax error
* Fixed syntax error
* Code clean-up: Handle exceptions, logs and formatting
Code formatted according to ORT coding guidelines.
* remove debug print from pybind code
* updated docs with ops and models
* formatting prints
* Added default values for c and j for openvino
* Overriding the values set for c and j to be 1
* BACKEND_OPENVINO should be empty if openvino is not in build
* Overriding c value with default for perftest
* fix VAD-M device string bug
* Add IE error details to exceptions
* Use IE specific device names in EP
* Add VAD-F (FPGA) device support
* Removed unecessary libraries from whl package
* Code changes for Windows compatibility
* Add VAD-F option to python API
* [revert before merge] cmake changes for RC
* Enable Windows build in CMake
* Unset macro OPTIONAL for windows builds
inference_engine.hpp's include chain defines a macro 'OPTIONAL'
which conflicts with onnx project's headers when using MSVC. So
would need to explictly unset it for MSVC.
* Use a single copy of plugin/IE::Core
Defined as a static member in Backend manager
* Remove restriction of single subgraphs for myriad
* Passed subgraph name to Backend to enhance log statements
* Disabled zero dimension conditions
* Disabled concat to remove zero dims
* Enabled building ngraph as part of ORT
* Removed serializing and added versioning
* Fix CPU_FP32 unit tests
* Removed unecessary condition
* add ngraph.so.0.0 to .whl
* Check for zero dimensions only for inputs and outputs
* Restrict loading only 10 subgraphs on myriad
* Build ngraph.dll within UEP. Doesn't link yet
* Rename Linux included libngraph.so to libovep_ngraph.so
Renames locally built libngraph.so containing ONNX importer to
libovep_ngraph.so in order to avoid linkage conflicts with
libngraph.so supplied by OpenVINO binary installer.
Applies only for Linux builds.
* use output_name cmake properties for lib name
* fix .so name format in lib_name.patch
* CMake code cleanup
* Rename WIN32 included ngraph.dll to ovep_ngraph.dll
To avoid conflict with ngraph.dll distributed by openvino.
* Added myriad config for networks without 4 dimensions
* Loading the 10 max clusters for inference on myriad
* Refactor code and add Batching support
Encapsulate subgraph settings into context structs.
Add batching support for completely supported models.
* Disabled some broken tests
* use input_indexes to avoid batch-checking initializers
* Avoid static initialization order error on WOS
* Added candy to broken tests
* InternalCI changes for 2020.2
* Updated DLDT instructions
* Unsaved changed in install_openvino.sh
* Changes after manual check
* Remove custom ngraph onnx_import build for WOS
ONNX Importer on WOS does not have protobuf issue.
* Remove FP32ToFP16 ngraph pass
This conversion is performed implicitly within IE.
* Surround debug logic by #ifndef NDEBUG
* remove invalid TODO comments
* removed references to ngrpah-ep
* clang-formatting
* remove commented code
* comment edits
* updating copyright year to that of first OpenVINO-EP release
* remove redundant log msg
* Modified operator and topology support
* Update build instructions
* doc formatting
* Fixed clip unit tests
* Revert "Remove FP32ToFP16 ngraph pass"
This reverts commit ec962ca5f315a5658ad980e740196f19de2639c1.
* Applying FP16 transformation only for GPU FP16
* Fixed GPU FP32 python tests
* automatically use full protobuf
* disable onnxrt server for now
* Disabled upsample
* update dockerfile instructions
* Removed MO paths and added ngraph path
* Remove OVEP from ORT Server docs
Will put it back in after validation
* Updated path to Ngraph lib
* Disabled Resize and some other python tests
* Removed unnecesary header files
* Use commit SHA to fetch ngraph repo
* Avoid un-needed file changes due to version update
* Fixed clip tests
* Fixed Pow, max and min onnx tests
* build.md doc typo
* Update cmake patch command for ngraph src
* remove dead cmake code for onnxruntime_USE_OPENVINO_BINARY
* use spaces instead of tab
* remove commented code
* Add info about protobuf version
* edit debug env var and enable for WIN32
* specify only version tag of 2020.2 for dockerbuilds
* remove unnecessary file changes
* Pass empty string as default argument to C# tests
* Use ${OPENVINO_VERSION} to name openvino install directory in CI builds
* Enabled unnecessarily disabled tests
* Fixed ngraph protobuf patch
* Fixed error in protobuf patch
* Revert "Use ${OPENVINO_VERSION} to name openvino install directory in CI builds"
This reverts commit 89e72adb8bf3b9712f5c81c5e13fe68c6c0df002.
* Remove unsetting OPTIONAL macro
This is no longer used in recent ONNX update onnx/onnx@da13be2,
so this unset workaround is no longer necessary.
* Use a null string default argument for C# API
* Set OpenVINO version yml files and pass to CI Docker builds
Git Tag info for DLDT as well as install directory are set
using this value.
This reverts commit 9fa9c20348ed72ae360a95c98e9b074d2f9fafc5.
* Documentation: recommendation and instructions for disabling ORT graph optimizations
* more doc updates
* Reduced the number of models according to CI time constraints
Co-authored-by: ynimmaga <yamini.nimmagadda@intel.com>
Co-authored-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
Co-authored-by: Mikhail Treskin <mikhail.treskin@intel.com>
Co-authored-by: mbencer <mateusz.bencer@intel.com>
Co-authored-by: Aravind <aravindx.gunda@intel.com>
Co-authored-by: suryasidd <48925384+suryasidd@users.noreply.github.com>
|
||
|
stevenlix
|
2332a93db0
|
Update onnx-tensorrt parser (#3369)
* sync onnx-tensorrt parser and update TensorRT doc * remove --msvc_toolset 14.16 in tensorrt ci pipeline |
||
|
Pranav Sharma
|
435f014d71
|
Add support for sessions to share a global threadpool. (#3177)
* Add support for sessions to share a global threadpool. * Fix build issues * Add tests, fix build issues. * Added some documentation * Fix centos issue when threadpools become nullptr due to 1 core. * Fix mac and x86 build issues * Address some PR comments * Disabled test for android, added few more tests and addressed more PR comments. * const_cast |
||
|
stevenlix
|
f4a5d17294
|
Upgrade to CUDA10.2 for TensorRT (#3084)
* Switch to CUDA10.2 * Update win-gpu-tensorrt-ci-pipeline.yml * Update win-gpu-tensorrt-ci-pipeline.yml * remove dynamic_shape * update onnx-tensorrt submodule * check if input shape is specified for TensorRT subgraph input and enable some TensorRT unit tests * fix format issue * add shape inference instruction for TensorRT * update according to the reviews * Update win-gpu-tensorrt-ci-pipeline.yml |
||
|
stevenlix
|
da653ccdac
|
Upgrade TensorRT to version 7.0.0.11 (#2973)
* update onnx-tensorrt submodule to trt7 branch * add fp16 option for TRT7 * switch to master branch of onnx tensorrt * update submodule * update to TensorRT7.0.0.11 * update to onnx-tensorrt for TensorRT7.0 * switch to private branch due to issues in master branch * remove trt_onnxify * disable warnings c4804 for TensorRT parser * disable warnings c4702 for TensorRT parser * add back sanity check of shape tensort input in the parser * disable some warnings for TensorRT7 * change fp16 threshold for TensorRT * update onn-tensorrt parser * fix cycle issue in faster-rcnn and add cycle detection in GetCapability * Update TensorRT container to v20.01 * Update TensorRT image name * Update linux-multi-gpu-tensorrt-ci-pipeline.yml * Update linux-gpu-tensorrt-ci-pipeline.yml * disable rnn tests for TensorRT * disable rnn tests for TensorRT * disabled some unit test for TensorRT * update onnx-tensorrt submodule * update build scripts for TensorRT * formating the code * Update TensorRT-ExecutionProvider.md * Update BUILD.md * Update tensorrt_execution_provider.h * Update tensorrt_execution_provider.cc * Update win-gpu-tensorrt-ci-pipeline.yml * use GetEnvironmentVar function to get env virables and switch to Win-GPU-2019 agent pool for win CI build * change tensorrt path * change tensorrt path * fix win ci build issue * update code based on the reviews * fix build issue * roll back to cuda10.0 * add RemoveCycleTest for TensorRT * fix windows ci build issues * fix ci build issues * fix file permission * fix out of range issue for max_workspace_size_env |
||
|
Changyoung Koh
|
7666d130e5 | Rename MKL-DNN to DNNL to fix broken link (#2730) | ||
|
Faith Xu
|
bb7f43ee91
|
Documentation update: build instructions (#2636)
* Spacing fix for code block * Update instructions Include java, acl, and nn api instructions on build page * Update build instructions to link to build.md * typo * Update build instructions to link to build.md * Include other minor build.md page updates * Update CUDA version * Fix dockerfile links |
||
|
daquexian
|
62de8fa841 | Update docs for Android NNAPI EP (#2586) | ||
|
stevenlix
|
293b15480b |
Add dynamic shape support in TensorRT execution provider (#2450)
* remove onnx-tensorrt submodule * add new onnx-tensorrt submodule (experiment) for trt6 * update engine build for trt6 * update compile and compute for tensorrt6.0 * Update tensorrt_execution_provider.cc * Update tensorrt_execution_provider.cc * Update tensorrt_execution_provider.cc * Update tensorrt_execution_provider.cc * switch to onnx-tensorrt master for TensorRT6' * Update tensorrt_execution_provider.cc * Handle dynamic batch size and add memcpy in TensorRT EP * update test cases * Update tensorrt_execution_provider.cc * update onnx-tensorrt submodule * Update Dockerfile.ubuntu_tensorrt * Update Dockerfile.ubuntu_tensorrt * Update run_dockerbuild.sh * Update run_dockerbuild.sh * Update install_ubuntu.sh * Update concat_op_test.cc * Update tensorrt_execution_provider.cc * Upgrade TensorRT to version 6.0.1.5 * Update onnxruntime_providers.cmake * Update CMakeLists.txt * Update reduction_ops_test.cc * Update install_ubuntu.sh * Update Dockerfile.ubuntu_tensorrt * Update Dockerfile.tensorrt * Update BUILD.md * Update run_dockerbuild.sh * Update install_ubuntu.sh * Update onnxruntime_providers.cmake * Update install_ubuntu.sh * Update install_ubuntu.sh * Update gemm_test.cc * Update gather_op_test.cc * Update CMakeLists.txt * Removed submodule * update onnx-tensorrt submodule * update header file * Removed submodule * add submodule onnx-tensorrt kevin's branch shape-test' * add debugging code * Update tensorrt_execution_provider.cc * Update tensorrt_execution_provider.cc * merge master * Removed submodule * update onnx-tensorrt submodule * add more changes for dynamic shapes * Update tensorrt_execution_provider.cc * update for dynamic shape * update dynamic shape processing * fix logger issue * remove submodule onnx-tensorrt * add submodule onnx-tensorrt * add env variable min_subgraph_size * remove redundency * update document * use onnxruntime::make_unique * fix multi-run issue * remove some tests to save CI build time * Add dynamic shape test * Update TensorRT-ExecutionProvider.md * Add example of running Faster R-CNN model on TensorRT EP * Add more details on env variables * update environment variables * Update tensorrt_basic_test.cc * Update model tests * Update tensor_op_test.cc * remove --use_full_protobuf * Update build.py |
||
|
Sreekanth Yalachigere
|
31ea11a696 |
Renaming MKL-DNN as DNNL (#2515)
* DNNL: Moving Files to rename file names * DNNL name change * azure pipeline updated * disable ceil/dialation and enable Opset10 * disable ceil/dialation tests in Python * mlperf_ssd_resnet34_1200 disabled |
||
|
KeDengMS
|
60208463a9
|
[NupharEP] Enable parallel schedule (#2505)
* [NupharEP] Enable parallel schedule * Update TVM with the fix to TVM threadpool to use OpenMP if possible * Add parallel schedule when trying to vectorize With this change, BERT squad perf on a 4-core (8 HT) CPU goes from 187ms to 150ms * Address CR, docs and cmake update * Doc fix * Fix mkl * Fix TVM windows build when using mklml |
||
|
Patrick Foley
|
151075790d |
[OpenVINO-EP] Update to latest version: OpenVINO 2019 R3.1 (#2308)
* Updates OpenVINO EP to latest version: 2019 R3.1 * Reviews fixed * Update Dockerfile.openvino * Addressed PR comments and disabled model tests temporarily * Update Dockerfile.ubuntu_openvino |