Commit graph

56 commits

Author SHA1 Message Date
S. Manohar Karlapalem
f7cf703d10
[OpenVINO-EP] Optimize MCR Docker image size (#3732)
* updated dockerfile.openvino

* Group all RUN commands and add a 'cd WORKDIR' betwen each

* Update doc with installer and build info

Highlight usage of Online installer package.
Specify --rm option during docker build to avoid caching layer.

Co-authored-by: avidiyal <akhila.vidiyala@intel.com>
2020-04-29 00:08:15 +08:00
George Wu
c23b484275 add missing deps in Dockerfile.openvino 2020-04-26 22:02:48 -07:00
S. Manohar Karlapalem
6d4f2f5bf9
OpenVINO EP v2.0 (#3585)
* Added FP16 transformations

* Revert "Added CMAKE_BUILD_TYPE to make building dynamic"

This reverts commit d3e17af1af655cfdc4d2fec33f52055caa525e85.

* Added FP16 transformations for FP16 builds

* Backend logic cleanup

Cleans the backend(intel_graph.*) code in the following ways:-

1. Minimize global usage: Since all the IR graphs need to be
re-generated on every Infer, it is bad practice to rely on globals
for their saving and usage as there would be multiple readers and
writers to the same global variable leading to incorrect usages or
contentions. This change replaces globals with locals where possible.
 This change also fixes an existing bug with due to
incorrect global usage.

2. Remove all unused functions.

3. Remove all unused headers and prepocessor directives.

* removed commented out code

* Disabled default optimization for Intel EP

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Fix missed plugins.xml for python bindings

* Fixed the build after latest master changes

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Disabled unsupported ops for accelerators

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Added some more disabled ops

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Added environment variable to enable debugging

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Added more debug statements

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Fixed unsupported ops list for GPU and VPU

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Fixed unsqueeze unit tests

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Added error message to the status

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Overwrite Model proto with shape info from data

Overwrites the shape info of Model proto with the shape from
actual input data. Needed for inferring models with Dynamic
shapes.

* Removed print statement and disabled where op

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Disabled Reshape with Empty initializer

* Added more debug statements for 1P

* Don't allow 1D inputs with symbol for dimension

* Disabled some 3rd phase ops

* Disabled split and added zero dimension check for OutputDefs

* Cleanup zero dimensionality check

* Added different data type check for inputs and initializers

* Added conditions for Mod, Cast and Pad

* Removed unused variable

* Disabled scan and added conditions for squeeze

* Added changes for fixing all C++ unit tests

* Implements Backend Manager class for caching

Backend Manager provides a layer of indirection between EP interface
and OV backend that provides caching services for models with
symbolic dims in input shapes.

* clean up commented blocks

* clang-formatting

* Read I/O type info from ModleProto

Read the tensor element type information from ModelProto object,
as FusedNode is no longer available.

* code cleanup

* clang-formatting

* Added print statement for jenkins

* Disabled some python tests

* Changed the path of convert fp32 to fp16 hpp

* Added conditions for BatchNorm in GetCapability

* Fixed failed tests

* Revert "Added conditions for BatchNorm in GetCapability"

This reverts commit c3c28c3b00d27892c42546b35dacdd807a48ee90.

* Added Intel to onnxruntime backends

* pick up vars set by OV package setupvars.sh

* Added conditions for Identity

* remove a few cout prints

* Added conditions for GPU_FP32 unit tests

* Revert "pick up vars set by OV package setupvars.sh"

This reverts commit 8199e029c03eae21a1a7ef6bfdc93d00e5d0198b.

* Commented out fatal message for protobuf

* Might need to be removed

* Add interface class for current backend

* moved common logic to base class

* simplified cpu backend

* Removed unused headers

* use vectors to save i/o tensors for windows compatibility

* move utils fxns to backend_utils namespace

* rename ov_backend to ibackend

* Factory pattern for backend creation

* rename CPU backend to Basic backend

* renamed to vad-M and added to factory list

* Added conditions for VPU

* Added print statements

* Changed the logic for checking for symbolic shapes

* Modified logic for zero dimension check

* Removed VPU single dimension condition

* Removed comments

* Modified logic in DimensionCheck method

* Remove legacy OpenVINO EP

Remove all the legacy code for OpenVINO EP. UEP code will take its
place going forward.

This change does NOT remove OVEP files in the following areas asa
they will be reused by UEP:-
1. Documentation: All .md files
2. Docker releated files
3. Python bindings
4. Java bindings
5. C# bindings
6. ORT Server
7. CI pipeline setup files

* Rename Intel EP to OpenVINO EP

* Added unique names to the subgraphs

* Removed subgraphs with only constant inputs

* Modified subgraph partitioning algorithm to remove const input subgraphs

* Apply suggestion to onnxruntime/core/providers/openvino/openvino_execution_provider.cc

* Tracking output names to fix the output order bug

* Changed output names to a unordered map

* Modified logic to check for symbolic input shapes

* Fixed a bug in Reshape check

* Added empty model path to Model constructor

* Made necessary changes to cmake to build from the binary package

* Changed INTEL_CVSDK_DIR to INTEL_OPENVINO_DIR

* Enable dyn device selection with C++ API

* Added Round operator to unsupported list

* Modified subgraph partition logic for MYRIAD

* Removed supported ops from the list

* Enable dyn dev selection in Py API's

* Add documentation for dynamic device selection

* Use MYRIAD || HDDL instead of VPU

* Removed temporary cast of Int64 to FP32

* Disabled unit Tests for CPU_FP32 and GPU_FP32

* Removed default "CPU" from unit tests to allow overriding

* Removed ops Concat, Squeeze, Unsqueeze from unsupported list

* Get the device id from info

* Removed overwriting device_id and precision

* Enabled ConvTranspose and EyeLike

* Reordered unsupported ops in alphabetical order

* Fixed syntax error

* Fixed syntax error

* Code clean-up: Handle exceptions, logs and formatting

Code formatted according to ORT coding guidelines.

* remove debug print from pybind code

* updated docs with ops and models

* formatting prints

* Added default values for c and j for openvino

* Overriding the values set for c and j to be 1
* BACKEND_OPENVINO should be empty if openvino is not in build

* Overriding c value with default for perftest

* fix VAD-M device string bug

* Add IE error details to exceptions

* Use IE specific device names in EP

* Add VAD-F (FPGA) device support

* Removed unecessary libraries from whl package

* Code changes for Windows compatibility

* Add VAD-F option to python API

* [revert before merge] cmake changes for RC

* Enable Windows build in CMake

* Unset macro OPTIONAL for windows builds

inference_engine.hpp's include chain defines a macro 'OPTIONAL'
which conflicts with onnx project's headers when using MSVC. So
would need to explictly unset it for MSVC.

* Use a single copy of plugin/IE::Core

Defined as a static member in Backend manager

* Remove restriction of single subgraphs for  myriad

* Passed subgraph name to Backend to enhance log statements

* Disabled zero dimension conditions

* Disabled concat to remove zero dims

* Enabled building ngraph as part of ORT

* Removed serializing and added versioning

* Fix CPU_FP32 unit tests

* Removed unecessary condition

* add ngraph.so.0.0 to .whl

* Check for zero dimensions only for inputs and outputs

* Restrict loading only 10 subgraphs on myriad

* Build ngraph.dll within UEP. Doesn't link yet

* Rename Linux included libngraph.so to libovep_ngraph.so

Renames locally built libngraph.so containing ONNX importer to
libovep_ngraph.so in order to avoid linkage conflicts with
libngraph.so supplied by OpenVINO binary installer.
Applies only for Linux builds.

* use output_name cmake properties for lib name

* fix .so name format in lib_name.patch

* CMake code cleanup

* Rename WIN32 included ngraph.dll to ovep_ngraph.dll

To avoid conflict with ngraph.dll distributed by openvino.

* Added myriad config for networks without 4 dimensions

* Loading the 10 max clusters for inference on myriad

* Refactor code and add Batching support

Encapsulate subgraph settings into context structs.

Add batching support for completely supported models.

* Disabled some broken tests

* use input_indexes to avoid batch-checking initializers

* Avoid static initialization order error on WOS

* Added candy to broken tests

* InternalCI changes for 2020.2

* Updated DLDT instructions

* Unsaved changed in install_openvino.sh

* Changes after manual check

* Remove custom ngraph onnx_import build for WOS

ONNX Importer on WOS does not have protobuf issue.

* Remove FP32ToFP16 ngraph pass

This conversion is performed implicitly within IE.

* Surround debug logic by #ifndef NDEBUG

* remove invalid TODO comments

* removed references to ngrpah-ep

* clang-formatting

* remove commented code

* comment edits

* updating copyright year to that of first OpenVINO-EP release

* remove redundant log msg

* Modified operator and topology support

* Update build instructions

* doc formatting

* Fixed clip unit tests

* Revert "Remove FP32ToFP16 ngraph pass"

This reverts commit ec962ca5f315a5658ad980e740196f19de2639c1.

* Applying FP16 transformation only for GPU FP16

* Fixed GPU FP32 python tests

* automatically use full protobuf

* disable onnxrt server for now

* Disabled upsample

* update dockerfile instructions

* Removed MO paths and added ngraph path

* Remove OVEP from ORT Server docs

Will put it back in after validation

* Updated path to Ngraph lib

* Disabled Resize and some other python tests

* Removed unnecesary header files

* Use commit SHA to fetch ngraph repo

* Avoid un-needed file changes due to version update

* Fixed clip tests

* Fixed Pow, max and min onnx tests

* build.md doc typo

* Update cmake patch command for ngraph src

* remove dead cmake code for onnxruntime_USE_OPENVINO_BINARY

* use spaces instead of tab

* remove commented code

* Add info about protobuf version

* edit debug env var and enable for WIN32

* specify only version tag of 2020.2 for dockerbuilds

* remove unnecessary file changes

* Pass empty string as default argument to C# tests

* Use ${OPENVINO_VERSION} to name openvino install directory in CI builds

* Enabled unnecessarily disabled tests

* Fixed ngraph protobuf patch

* Fixed error in protobuf patch

* Revert "Use ${OPENVINO_VERSION} to name openvino install directory in CI builds"

This reverts commit 89e72adb8bf3b9712f5c81c5e13fe68c6c0df002.

* Remove unsetting OPTIONAL macro

This is no longer used in recent ONNX update onnx/onnx@da13be2,
so this unset workaround is no longer necessary.

* Use a null string  default argument for C# API

* Set OpenVINO version yml files and pass to CI Docker builds

Git Tag info for DLDT as well as install directory are set
using this value.

This reverts commit 9fa9c20348ed72ae360a95c98e9b074d2f9fafc5.

* Documentation: recommendation and instructions for disabling ORT graph optimizations

* more doc updates

* Reduced the number of models according to CI time constraints

Co-authored-by: ynimmaga <yamini.nimmagadda@intel.com>
Co-authored-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
Co-authored-by: Mikhail Treskin <mikhail.treskin@intel.com>
Co-authored-by: mbencer <mateusz.bencer@intel.com>
Co-authored-by: Aravind <aravindx.gunda@intel.com>
Co-authored-by: suryasidd <48925384+suryasidd@users.noreply.github.com>
2020-04-24 04:06:02 -07:00
stevenlix
2ab78c5da1
Update TensorRT parser (#3650)
* update onnx-tensorrt submodule

* add more model dumping point

* update trt kernel name and docker readme file

* fix minior issues

* fix format issue

* update onnx-tensorrt submodule

Co-authored-by: stevenlix <stevenlix>
2020-04-23 20:51:44 -07:00
manashgoswami
044c466158
Updated tags for v1.2.0 release (#3386)
Updated the tags in the table to reflect the new images for Release v1.2
2020-03-31 14:54:56 -07:00
Changming Sun
0fceb33288
Fix onnxruntime server docker file build failure (#3219)
1. Fix onnxruntime server docker file build failure. Tested with the notebook in ONNX tutorial, it works well.
2. Delete the docker files for the other EPs, because currently they don't work and I don't have enough time to update them.
2020-03-15 14:46:46 -07:00
George Wu
584ba71485
TensorRT dockerfile updates (#3016)
* change npy installation

* update trt base image version

* update build instructions for arm64/jetson
2020-02-13 14:34:00 -08:00
Qing
723cf83793 Update Ubuntu & TensorRT version in README (#2820)
Dockerfile.tensorrt is using nvcr.io/nvidia/tensorrt:19.09-py3 as base Image, update Ubuntu and TensorRT version according to
https://docs.nvidia.com/deeplearning/sdk/tensorrt-container-release-notes/rel_19-09.html#rel_19-09
2020-01-13 14:37:32 -08:00
Mark
79847f39b3 Fix file not found error during docker build. (#2569) 2019-12-07 08:49:47 -08:00
Sreekanth Yalachigere
31ea11a696 Renaming MKL-DNN as DNNL (#2515)
* DNNL: Moving Files to rename file names

* DNNL name change

* azure pipeline updated

* disable ceil/dialation and enable Opset10

* disable ceil/dialation tests in Python

* mlperf_ssd_resnet34_1200 disabled
2019-12-03 07:34:23 -08:00
KeDengMS
c1be615c45
[NupharEP] refine parallel schedule control (#2514)
* [NupharEP] Add parallel schedule to JIT function name
Update Nuphar docker to use Python 3.6 and ubuntu 18.04

* Update notebook

* Avoid JIT cache file name conflict
2019-12-02 17:40:51 -08:00
KeDengMS
e0b51e4d95
Symbolic shape inference improvements: (#2460)
* Symbolic shape inference improvements:
- add a mode to guess unknown ops' output rank
- add support for GatherND
- add support for If
- fix a bug in get_int_values when then tensor rank > 1D, by treating it as no sympy data
- add symbol to literal merge when ONNX silently merges dims
- fix a bug in Concat when input dim is 0
- fix a bug in ConstantOfShape that computed dim is not updated
- add support for dynamic shape in ConstantOfShape
- fix a bug in Loop output shape that loop iterator dim is not inserted at dim 0
- add support for dynamic padding in Pad
- add support for dynamic shape in Reshape
- add support for Resize with opset > 10, by treating output dims as dynamic
- fix a bug in Slice when starts/ends are dynamic
- restrict input model to opset 7 and above
- make output model optional to avoid disk write when testing

Run model tests for symbolic shape inference

Reduce 2GB docker image size of nuphar
2019-11-25 10:31:13 -08:00
KeDengMS
aa7c79eac9 [NupharEP] Update notebook and docker image (#2416)
Add BERT squad in Nuphar tutorial
Enhance speed comparsion readability
2019-11-18 10:38:14 -08:00
avidiyal
3d3cf0e159 Openvino EP R3.1 onnxrt server (#2357)
* onnxrt server with OVEP

* onnxrt server with OVEP

* Update Dockerfile.server.openvino

* onnxrt server OVEP fix reviews

* onnxrt server OVEP fix reviews
2019-11-11 12:22:19 -08:00
George Wu
ba0e7daf20
update dockerfiles/README (#2336) 2019-11-06 16:54:10 -08:00
George Wu
06a6d74a67
update ngraph dockerfile. add python lib location to LD_LIBRARY_PATH for cuda/tensorrt Dockerfiles. (#2330) 2019-11-06 11:29:55 -08:00
Vinitra Swamy
ace19129b9 MCR Docker Images v1.0.0 refresh (#2302)
* update dockerfile table with new MCR tags

* add new openvino dockerfiles to table
2019-11-05 22:06:47 -08:00
Patrick Foley
151075790d [OpenVINO-EP] Update to latest version: OpenVINO 2019 R3.1 (#2308)
* Updates OpenVINO EP to latest version: 2019 R3.1

* Reviews fixed

* Update Dockerfile.openvino

* Addressed PR comments and disabled model tests temporarily

* Update Dockerfile.ubuntu_openvino
2019-11-05 19:55:46 -08:00
George Wu
aa041026e3
update Dockerfile.openvino (#2286)
* install miniconda before openvino installation. add networkx, defusedxml dependencies.

* apt-get update

* apt-get update

* merge Intel changes.
2019-10-30 13:58:24 -07:00
zhijxu
8dabe0502b merge two RUN to avoid making docker image too larger 2019-10-30 09:57:56 -07:00
zhijxu
0b88eff43a add docker file to build onnxruntime with different execution providers 2019-10-28 10:03:18 -07:00
zhijxu
3926ce33f4 add --parallel to speed up compiling source code 2019-10-28 10:03:18 -07:00
zhijxu
4eda08534f make smaller step in Dockerfile
so image layers can be reused as much as possible
2019-10-28 10:03:18 -07:00
Pranav Sharma
69970d1f2a
Include the new Privacy.md file in all release packages. (#2200) 2019-10-20 07:58:36 -07:00
Klein Hu
2bf1778a5c Fix ORTSRV docker build (#2160)
* Fix ORTSRV docker build

* More Dockerfile changes
2019-10-17 21:10:21 -07:00
Faith Xu
ec136ac60f
Documentation Refresh (#1990)
Various documentation updates, primarily for EP and main readme page
2019-10-15 15:58:02 -07:00
stevenlix
544e53e24e Update TensorRT to version 6.0.1.5 (#1966)
* remove onnx-tensorrt submodule

* add new onnx-tensorrt submodule (experiment) for trt6

* update engine build for trt6

* update compile and compute for tensorrt6.0

* Update tensorrt_execution_provider.cc

* Update tensorrt_execution_provider.cc

* Update tensorrt_execution_provider.cc

* Update tensorrt_execution_provider.cc

* switch to onnx-tensorrt master for TensorRT6'

* Update tensorrt_execution_provider.cc

* Handle dynamic batch size and add memcpy in TensorRT EP

* update test cases

* Update tensorrt_execution_provider.cc

* update onnx-tensorrt submodule

* Update Dockerfile.ubuntu_tensorrt

* Update Dockerfile.ubuntu_tensorrt

* Update run_dockerbuild.sh

* Update run_dockerbuild.sh

* Update install_ubuntu.sh

* Update concat_op_test.cc

* Update tensorrt_execution_provider.cc

* Upgrade TensorRT to version 6.0.1.5

* Update onnxruntime_providers.cmake

* Update CMakeLists.txt

* Update reduction_ops_test.cc

* Update install_ubuntu.sh

* Update Dockerfile.ubuntu_tensorrt

* Update Dockerfile.tensorrt

* Update BUILD.md

* Update run_dockerbuild.sh

* Update install_ubuntu.sh

* Update onnxruntime_providers.cmake

* Update install_ubuntu.sh

* Update install_ubuntu.sh

* Update gemm_test.cc

* Update gather_op_test.cc

* Update CMakeLists.txt

* Removed submodule

* update onnx-tensorrt submodule

* Add Ubuntu18.04 build option

* Add Ubuntu18.04 build option

* Add Ubuntu18.04 build option

* Add Ubuntu18.04 build option

* Remove redundency

* Fix issue that it does not add memcopy node correctly if some nodes fall back to CUDA EP.
e.g. after partition, there's TRT_Node -> Cuda_node (with CPU memory expected), we still need to add memcpy node between them.

* update for Trt Windows build

* Update onnxruntime_providers.cmake

* Disable opset11 tests on TensorRT

* Update pad_test.cc

* Update build.py

* update scripts for ubuntu18.04

* Disable warning for Windows build
2019-10-06 10:40:53 -07:00
Vinitra Swamy
7c77a01ce7
OpenVINO dockerfile for ORT (#1892)
* dockerfile openvino updates
2019-10-02 14:04:04 -07:00
Vinitra Swamy
2a21df2309
Updates to CUDA and TensorRT dockerfiles for v0.5.0 (#1731)
* updates to cuda and tensorrt dockerfiles for v0.5.0

* add table of build tags
2019-09-13 14:16:47 -07:00
KeDengMS
58fe5a6bf1
Enable Nuphar docker build, and reinstate Nuphar tests (#1757)
Enable Nuphar EP docker build
Revert back to LLVM 6.0.1
Reinstate disabled Softmax tests caused by LLVM 8.0.1
Reinstate Nuphar Python test due to stale sympy version
Increase build timeout of Linux CI
2019-09-05 08:50:48 -07:00
manashgoswami
3d44c55092 Updated docs related to base images (#1753)
* Update README.md

* Update onnx-inference-byoc-gpu-cpu-aks.ipynb

* Update README.md
2019-09-04 10:33:41 -07:00
Hector Li
dc9c89546d
Update the docker file for OpenVINO (#1741)
Update the docker file for OpenVINO which is used for AML
2019-08-30 22:32:24 -07:00
rakelkar
0f7c01b49b Use exec form of ENTRYPOINT for docker server (#1690)
* Use exec form of ENTRYPOINT for docker server

# Issue
The entrypoint currently uses the shell form - this prevents users from passing in any cmdline arguments... also passing a model_path in means the server only works in the envvar is set... however this is not what the error message says!
```
$ docker run -v /home/rakelkar/try/onnxzoo/style:/mnt/models -it   mcr.microsoft.com/onnxruntime/server --model_path /mnt/models/model.onnx
Version: local_build
Commit ID: default

model_path must be the location of a valid file
Allowed options:
  -h [ --help ]               Shows a help message and exits
  --log_level arg (=info)     Logging level. Allowed options (case sensitive): 
                              verbose, info, warning, error, fatal
  --model_path arg            Path to ONNX model
  --address arg (=0.0.0.0)    The base HTTP address
  --http_port arg (=8001)     HTTP port to listen to requests
  --num_http_threads arg (=4) Number of http threads
  --grpc_port arg (=50051)    GRPC port to listen to requests
```
# Fix
1. remove the env var
2. use the exec form

* Update readme to use model_path arg
2019-08-29 10:18:08 -07:00
Faith Xu
d9cdf4b4ed
Doc updates (#1522)
* Updates

* Remove preview texts

* Update README.md

* Updates

* Update README.md

* Update README.md

* Minor wording update

* Update README.md

* Update doc on CUDA version

* revert update

* Update readme for issue #1558

* Clean up example section

* Cosmetic updates

- Add a index of build instructions for browsability
- Update build CUDA version from 9.1 to 10

* Fix broken link

* Update README to reflect upgrade to pip requirement

* Update CuDNN version for Linux Python packages

* Clean up content

Updated ordering and add table of contents

* Minor format fixes

* Move Android NNAPI under EP section

* Add link to operator support documentation

* Fix typo

* typo fix

* remove todo section
2019-08-27 21:31:19 -07:00
manashgoswami
6d783e8a07 Added license files in the base image (#1595)
* Update Dockerfile.openvino

* Update Dockerfile.cuda

* Update Dockerfile.cuda

* Update Dockerfile.openvino

* Update Dockerfile.cuda

* added ThirdParty notice file to base image.

* corrected license file name
2019-08-09 13:02:06 -07:00
S. Manohar Karlapalem
05bbb3065c [OpenVINO-EP] Update hardware branding of VAD-R as VAD-M (#1552)
Replaces all occurrences of VAD-R/VAD_R with VAD-M/VAD_M.
Aligns with the official hardware branding.
2019-08-05 15:28:46 -07:00
avidiyal
c0f927c57c docker updated to support openvino R1.1 (#1475)
* docker updated to support openvino R1.1

* Update README.md

Updated Readme to downlaod openvino R1.1
2019-07-24 13:16:23 -07:00
suryasidd
e9e777925f [OpenVINO-EP] Added support for OpenVINO R1.1 (#1438)
* Initial commit for OpenVINO R1

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Fixed MO dynamic shape error

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Add debug messages for failure

* Update install_openvino.sh script

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Try catch included.  Return type of Isgraphsupported function changed to void

* Removed error_msg variable and commented code

* formatting cleanup

* Added missing return statement

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Changed MO to be compatible with both R5 and R1

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Updated docker scripts to include openvino version number

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Ignore compiler warnings from external headers

* Updated dockerfiles

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Code cleanup using clang-format

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Suppress model optimizer info error

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Python code formatting using auto pep8

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Updated documentation

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
2019-07-19 00:52:15 -07:00
Colin Versteeg
5ee0f185dc Add GRPC support to ONNX Runtime Server (#1144)
* add grpc

* add-submodule

* Revert "add-submodule"

This reverts commit e35994b25035ce310a98909658582bff759ee358.

* fix submodule

* IT BUILDS

* Initial commit of prediction_service_impl.cpp

* Server builds and runs!

* add request id, health and reflection. GRPC is done

* enable channelz for monitoring

* GRPC unit tests

* clang format

* add unit tests

* Add function tests for GRPC

* add grpc to model_zoo_tests

* revert update protobuf to 3.7.0

* update submodules

* builds but runs some gflags tests which fail

* get build working

* confine build changes to onnxruntime_server.cmake

* update build files

* code reveiw comments

* Maik's code review comments

* update cares version to fix compilation issue

* update build to fix c-ares

* code review comments

* update cgmanifest.json

* remove extraneous file

* Klein comments.

* update ci based on discussions for go dependency

* fix tag issue

* fix build issues

* remove stray submodule

* update dockerfile and build script

* dynamic linking changes

* update build script

* code review comments

* update dockerfile

* update script for mount

* code review comments
2019-07-18 11:10:38 -07:00
avidiyal
859a57d781 Updated Dockerfile for OpenvinoEP (#1362)
* Updated Dockerfile for OpenvinoEP

Signed-off-by: avidiyal <akhila.vidiyala@intel.com>

* Changed the license

Signed-off-by: avidiyal <akhila.vidiyala@intel.com>

* resolving conflicts

* Reviews fixed
2019-07-17 14:52:59 -07:00
Vinitra Swamy
6b32c77804 Dockerfiles for TensorRT, CUDA, build from source (#922)
* dockerfile updates for BYOC scenario

* updates for 3 different build versions

* updating to remove libopenblas, python3, python3-pip

* Including LICENSE-IMAGE.txt for CUDA/TensorRT dockerfiles

* remove unnecessary cmake files

* fixing comment typo

* optimizing dockerfile.source as per review suggestions (not working currently)

* Optimizing dockerfiles with install_dependencies script

* update dockerfile with --cmake_extra_defines version number

* add &&\ for license copy lines

* updates, adding miniconda to path, reincluded clearing the pycache

* adding maintainer note

* update readme instructions

* update tensorrt versioning in dockerfile
2019-07-09 02:03:55 -07:00
NonStatic
9f9ff19bdc Copy shared library after build ORT Server (#1347) 2019-07-07 20:21:16 -07:00
RandySheriffH
671c15a56a
Treat attribute warning as non-error on cross compiling ARM (#1261)
* abandon attribute error on cross compiling

* install dep lib
2019-06-23 17:59:38 -07:00
Vinitra Swamy
3b71701f91
Update onnxruntime server docker file with ONNXRUNTIME_VERSION in cmake files (#1259) 2019-06-21 10:28:06 -07:00
Jorgen Thelin
ba25ea3643 Allow building Docker container based on a different git repo. (#1222)
- Introduce Docker build ARG `ONNXRUNTIME_REPO`
  to allow building Docker container based on a different git repo.

Example docker build command:

```bash
cd dockerfiles
docker build -t onnx-runtime \
  --build-arg ONNXRUNTIME_REPO=https://github.com/jthelin/onnxruntime \
  --build-arg ONNXRUNTIME_SERVER_BRANCH=my-branch \
  -f Dockerfile.server .
```

- Add a basic `.dockerignore` file, to cut down the number of filles passed into the Docker build context.
2019-06-20 09:55:42 -07:00
S. Manohar Karlapalem
8d15ffd8f5 Initial commit for OpenVINO Execution Provider (#935)
* Initial commit for OpenVINO Execution Provider

OpenVINO Execution Provider provides the interface for ONNX Runtime
applications to access Intel's hardware accelerators using Intel's
OpenVINO Toolkit.

* Fixed bug in GetCapability to disable custom ops

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Added OPENVINO ci pipeline

Added new pipeline for openvino provider,
made changes to support the docker build and
onnxruntime build with openvino.

Signed-off-by: Luis Daniel Castellanos <luis.daniel.castellanos@intel.com>

* Enabled all unit tests for OpenVINO EP

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Fixed syntax issue in run_docker_build.sh file

* Added missing default OPENVINO_VERSION

Default value for OPENVINO_VERSION env was
missing causing the build to fail

* Added install Model Optimizer deps step

* Fixed python unit tests and some tests from onnx_backend_test_series

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Fixed indentation bug

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Disabled some of the python backend tests for OpenVINO

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Disabled some model tests

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Remove Duplicate checks for openvino in build.py

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Modified GetCapability for FP16

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Disabled GPU FP32 tests that are not supported

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Convert modelProto to string and use it in compile

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Pass byte-array input args to MO

* Serialized ModelProto passed in-memory to MO

ModelOptimizer python module receives the serialized  ModelProto
in-memory.
Uses appropriate ONNX function to load the serialized bytes.

* Make Py_Finalize compatible with older python versions

Also, remove pFunc unassigned variable possibility.

* Fallback if input dims of Matmul is greater than 2

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* fixup: Device #define syntax

* Updated the documentation

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Enable dynamic dim value

* removed commented out code

* Added Dockerfile for openvino EP

Updated instructions on dockerfiles/README.md file

Signed-off-by: Luis Daniel Castellanos <luis.daniel.castellanos@intel.com>

* Disabled fp16_inception_v1 test

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Code formatting with clang-format

Uses style from the .clang-format file in root directory.

* fixup: docker tag and build error fixes

* Heuristics to automatically detect batching

Distributes slices from batch into parallel infer-request objects.

* Handle disabled tests in GetCapability

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Disabled average pool and max pool if ceil_mode is 1

Also dilations are not supported if they are greater than 1

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Disabled Unsqueeze int32 test

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* changes to fix output results bug

* Disabled a few C++ unit tests for MYRIAD FP16

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Manually revert '9fe162bb Enable dynamic dim value'

Reverts compile time setting of dynamic shape
Reverting manually due to significantly huge auto-revert conflicts.

* Fixed unused variable warning

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Disabled Mul test for GPU_FP16 due to accuracy issue

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* VPU documentation update

* Disabled inception_v1 for MYRIAD and HDDL

*Also disabled few C++ accuracy tests for HDDL

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* updates from upstream

* use the new CustomOpApis for I/O interfacing

* Pass initializers as subgraph meta-def inputs in GetCapability()

Requirement due to API changes introduced with PR# 1019.

* Remove obsolete functions

* Save indexes of graph inputs from fused_node info

Both inputs and initializers are passed as data inputs to the
infer function. To identify only inputs among them, save thier
index info from fused_node in Compile function.

* Documentation changes to enable VPU

* Fix VPU related changes in documentation

* Fix minor changes in documentation

* Fix VPU related changes in documentation

* Use Node.In/OutputDefs() to track graph inputs and outputs.

Don't use graph_viewer's GetInputs() or
GetInputsIncludingInitializers().

* Permit "SAME_UPPER" auto_pad attribute from MaxPool

* Disabled fp16_tiny_yolov2 in onnx model tests

* Updated documentation to include configuration guides for myriad and hddl

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Use 8 Infer requests only for VAD-R

* disable debug prints

* Clang-format source files

* Updated BUILD.md with OpenVINO R5 links

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Disabled same upper python tests

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Update test exclusion syntax

* Change path of install_onnx.sh

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Disable tiny_yolov2 in broken tests

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Revert "Change path of install_onnx.sh"

This reverts commit ba9db165f3be430f2aff1ef413299ed04637196a.
This change is only required for Intel internal CI pipeline until
the settings are matched with the upstream's CI pipeline.

* Added debug statements for debugging CI error

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Add --build_wheel to linux openvino pipeline

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Added -v option to onnx_test_runner for debugging

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Removed path change patch

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Added -c 1  to onnx_test_runner

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Refactor MO python invocation in separate function

Cleans up Model Optimizer python invocation check and conversion
logic. Invokes MO only once in GetCapability() and passes the
IR strings (xml and bin) to the Compiler as meta-def attributes.

* Add comments

* code cleanup and comments

* Code cleanup for GetCapability

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Removed unnecessary files

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Revert "Added -v option to onnx_test_runner for debugging"

This reverts commit d1dd70938a94d648df1a1dbbc2e48d0b97e49ec8.

* Revert "Added debug statements for debugging CI error"

This reverts commit b86d41afed2aa29c3508155d6f9c8d3a7263cc60.

* incorporate Status Code changes

* ComputeFunc returns Status::OK() on success

* Use test names to disable tests for MYRIAD and VAD-R

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Rename local identifiers from CNNNetwork to OpenVINO network

CNNNetwork is an OpenVINO's API class that represents more than
just convolutional neural networks (CNNs). Renaming helps to avoid
confusion that the API's only support CNN type models.

* Added error message if building on windows

* Removed duplicate option in Cmake
* Removed unnecessary parameters in activation_opt_test

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Refactor Map search and access logic for efficiently and cleanliness.

* use C++ style casts

* Use os.path.join for python directory path operations

* use C++ style casts

* EP classes should use onnxruntime namespace

* Clean up fixes from PR comments

* Don't explicitly shutdown Py interpreter

* Remove debug print statements

Prints will be re-enabled later with a logging mechanism with
debug/verbose printing options.

* Decrement ref counts for used pyObjects

* Restore build instructions for other compilers

Content under the "Using other compilers" section has been
accidentally deleted by a previous commit. Restoring back that
content from the latest upstream repo.

* CMake code cleanup

Code clean up, commenting and formatting of CMake code.

* Don't pass the unused device_info parameter to OpenVINOGraph ctor.

* Add support for multiple I/O data types

Adds support for the following tensor data types for graph inputs
and outputs:
1) float
2) float16
3) int32
4) int16
5) int8
6) uint16
7) uint8

* cleanup setup.py module list definition

* Deduce index of input using tracked input index map

Ignores initializers in case they are ordered before inputs.

* Removed debug statement in MO code

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* PR feedback

* Removed per_sample_tolerance for openvino
* Removed unnecessary disabled tests

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Removed debug function

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Disabled tiny_yolo_v2 due to accuracy issues

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Changed the disabled reason for broken tests

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Disabled Reshape with no input

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Python formatting with Autopep8

* Minor fix for MYRIAD devices

* Added zero dimension check

*Removed setting batch size for the network

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Set the threshold to larger value for MNIST

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Removed setting higher threshold in provider_test_utils

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>

* Check for --use_openvino in python wheel setup.py

Add openvino modules to the setup script for building the wheel
package only for --use_openvino a build option.

* Removed nullptr checks for GetNode()

Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
2019-06-18 08:58:53 -07:00
tmccrmck
cb25fd4d8a Updates ARMv7 Dockerfile to use newer CMake (#1017) 2019-05-14 13:33:44 -07:00
Harry Yang
2512b0ebeb Adding the onnxruntime Dockerfile and instructions (#968)
* Adding the onnxruntime Dockerfile and instructions

* Adding the onnxruntime server Dockerfile with build and instructions

* Specify more details in instruction

* Update dockerfiles/README.md

Co-Authored-By: tianchijushi <huaimingyang@hotmail.com>

* Update Dockerfile.server

Specify the onnxruntime server build branch during git clone
2019-05-08 18:24:16 -07:00
Vinitra Swamy
47171a076b
Addressing #956, deprecating old CPU and GPU quickstart dockerfiles (#987) 2019-05-08 10:40:01 -07:00
nivas-x86
7f14326dab nGraph EP documentation (#939) 2019-04-30 18:38:09 -07:00