This change adds a new execution provider powered by [DirectML](https://aka.ms/DirectML).
DirectML is a high-performance, hardware-accelerated DirectX 12 library for machine learning on Windows. DirectML provides GPU acceleration for common machine learning tasks across a broad range of supported hardware and drivers.
The DirectML execution provider is capable of greatly improving evaluation time of models using commodity GPU hardware, without sacrificing broad hardware support or requiring vendor-specific extensions to be installed.
**Note** that the DML EP code was moved verbatim from the existing WindowsAI project, which is why it doesn't yet conform to the onnxruntime coding style. This is something that can be fixed later; we would like to keep formatting/whitespace changes to a minimum for the time being to make it easier to port fixes from WindowsAI to ORT during this transition.
Summary of changes:
* Initial commit of DML EP files under onnxruntime/core/providers/dml
* Add cmake entries for building the DML EP and for pulling down the DirectML redist using nuget
* Add a submodule dependency on the Windows Implementation Library (WIL)
* Add docs under docs/execution_providers/DirectML-ExecutionProvider.md
* Add support for DML EP to provider tests and perf tests
* Add support for DML EP to fns_candy_style_transfer sample
* Add entries to the C ABI for instantiating the DML EP
Enable multi-device test for GPU
* Add build pipeline for TensorRT multi-GPU test
* Add code to disable fp16 test if hardware architecture not supported
* Add option to set the device id in onnx_test_runner for model tests
Remove gsl subodule and replace with a local copy of gsl-lite
Refactor for onnxruntime::make_unique
gsl::span size and index are now size_t
Remove lambda auto argument type detection.
Remove constexpr from fail_fast in gsl due to Linux not being happy.
Comment out std::stream support due to MacOS std lib broken.
Move make_unique into include/core/common so it is accessible for server builds.
Relax requirements for onnxruntime/test/providers/cpu/ml/write_scores_test.cc
due to x86 build.
Add ONNXRUNTIME_ROOT to Server Lib includes so gsl is recognized
1. remove sudo from the cleanup step for Linux so that we don't need the sudo access for vstsagent build user
2. a minor fix in the install_ubuntu.sh to make the image smaller for openvino
* Bump onnx to latest
Update onnx.in.proto with changes for SparseTensor.
* add temp skip tests
* remove passed tests from skip list
* skip more tests for new ops in opset 11
* skip crashing tests
* update handling of new attribute types sparse tensor and sparse tensors
* advance onnx commit and remove skip cpu_flaky_tests
* temporarily skip yolo3 model test due to resize opset10 shape inference regression
* update proto for onnxruntime server
* advance onnx commit further
1. Add openvino GPU nightly build pipeline, this test is running on Intel Up square Edge device. The device are host locally not from Azure VM. We persist a smaller model test data on Edge device.
2. Update the build condition for openvino GPU so it works for GPU_FP32, GPU_FP16
3. add option to install_ubuntu.sh to exclude the package used for nuphar, so that we can save some disk space as the Edge device usually have limited disk space.
Enable Nuphar EP docker build
Revert back to LLVM 6.0.1
Reinstate disabled Softmax tests caused by LLVM 8.0.1
Reinstate Nuphar Python test due to stale sympy version
Increase build timeout of Linux CI
- Fix the Windows end-to-end test in NuGet CI
- Skip the TestModelSerialization, because it is failing on Linux. Must be fixed before API is released for use. Owner is notified.
- Added python script for generating markdown doc from the registered opkernels.
- Made some conditional changes in the pybind to expose necessary python API
- Added some missing type-constraints in the op kernel registrations
* remove memory copy between CUDA and TRT
* add info to RegisterExecutionProvider input
* use new IDeviceAllocator for trt allocator
* remove SetDefaultInputsMemoryType from TRT EP
* remove onnx-tensorrt 5.0
* add submodule onnx-tensorrt branch 5.1
* remove redundancy
* Update transformer_memcpy.cc
* Update tensorrt_execution_provider.cc
* switch to TensorRT 5.1.5.0
* update python binding
* disable failed test case on TensorRT
* Update activation_op_test.cc
* upgrade to TensorRT container 19.06
* update according to feedback
* add comments
* remove tensorrt allocator and use cuda(gpu) allocator
* update onnx-tensorrt submodule
* change ci build cuda directory name
* Add MacOS leg of Python packaging job
* Update copy files source directory for Mac OS leg
* Add a task to display the binaries directories contents after build wheel creation
* Revert some changes
* Add task to log
* Update
* Remove unnecessary logs
Python script and necessary changes in the azure-pipelines yaml file to post the binary size data from NuGet package build. Currently only posted from CPU pipeline. GPU and other pipelines may be added as necessary.
* Update DNNLibrary
* Allow fp16 by default
* Add nnapi build in ci
* Fix nnapi ep after #1268
* Remove unused variables
* Support nnapi in onnx_test_runner
* Update DNNLibrary to fix tests
* Update build.py for android build support, solve conflict of
tools/ci_build/build.py
* Support non-ARM Android build, solve conflict of tools/ci_build/build.py
* Enable android test by x86_64 android emulator
* Add dnnlibrary/NNAPI support in build.py
* suppress the verbose adb output
* Remove debug logs
* Install cmake by pip
* Fix undefined host_protoc_path
* cmake==3.13.2 in pypi is actually 3.12.2, so install 3.13.2.post1 instead
* Fix Android ARM64 build
* Use android ndk r20 instead of r19c, fix conflicts in install_deps_android.sh
* sync onnx to get equal op with float support
* doc update
* fix test failure because of updated shape inference logic for roialign.
* filter consum test cases since it's not implemented yet.
* Add arm64 nocontribops pipeline
* minor fix
* Added new template for arm build -- disable all tests
* fix build command
* add arm64 flag for msbuild
* add arm leg as upstream dependency
* update platform to arm64 for msbuild
* remove test task from arm build
* remove ESRP signing of C# dlls in arm build
* Updated to work for both --arm and --arm64
* Make the cross compiling cmake flags symmetric
* Add dynamic check for /Wno-error flag, instead of extra build option
* remove extra full-stop
* replace log sinks
* limit headers to include dir
* first changes to do dynamic linking
* wip for using cxx api
* remove weird dangling dependency
* building with tests failing
* finish updating converters
* fix const
* intital introduction of typedef
* change logging to use spdlog
* get tests passing
* clang format
* map logging levels better
* clean up unused imports
* trent cr comments
* clang-format
* code review comments
* changing buffer use to reserve
* Dynamically link
* revert tvm
* update binary uploading
* catch exceptions by const-ref
* Revert "revert tvm"
This reverts commit 387676dd1018134d15eb71fa126f7caf94380800.
* fix typo
* update versioning of lib
* Update cuda for python wheels
* Update cuda for python wheels
* Update cuda for python wheels
* Update azure-pipelines-py-packaging.yml
* Update to cuda 10
* Only test win gpu
* Update cuda for python wheels
* Use manylinux2010 image to build linux python wheels
Allow wheels built to truly be compliant with a manylinux policy
* Simplify linux gpu pipeline
* Refactor win-gpu-ci-pipeline.yml
* Set cuda environment variables for testing and version
* Remove variables from starter script
* minor fix
* Add GPU Nuget pipeline
* Set DisableContribOps environment variable for Linux package tests
* Add ESRP tasks
* Add ESRP signing templates
* Test out hardcode value of ERSP
* Test out hardcode value of ERSP
* Test out hardcode value of ERSP
* Test out hardcode value of ERSP
* test variable expansion
* test variable expansion
* test variable expansion
* test variable expansion
* test variable expansion
* test variable expansion
* test out variable expansion
* test variable expansion
* test variable expansion
* test variable expansion
* test variable expansion
* test variable expansion
* test variable expansion
* test variable expansion
* test variable expansion
* update cpu pipeline to conditionally esrp sign
* Set C# GPU tests to run only if env var is set
* Refactor for easy parameter passing
* refactored esrp templates
* remove variables from template
* Add packaging variables back to pipelines
* update C# for cuda 10
* Merge vars ana parameters for gpu pipeline
* remove vars from mklml pipeline
* display envvars on terminal
* Clean up C# cuda tests, and upgrade to Cuda10
* Introduce CUDNN_PATH pipeline varaible
* YAML variable are always uppercased (not true with classic)
* Update C# GPU test to be more meaningful
* remove macos from gpu tests
* remove debugging info for DisableContribOps option
* Remove DisableContrib ops parameters -- use variables only
* Fix typo from = to -
* remove debug steps
* fix typo
* remove unused variable TESTONGPU from some templates
* clean up CUDA env setup scripts
* Remove CUDNN_PATH from setup_env_cuda.bat
* call install_onnx with relative path
* use caller path
* get abs path of current script
* process unicode char
* replace script name
* add suffix as part of match
* match only the end of path
* Initial commit for OpenVINO Execution Provider
OpenVINO Execution Provider provides the interface for ONNX Runtime
applications to access Intel's hardware accelerators using Intel's
OpenVINO Toolkit.
* Fixed bug in GetCapability to disable custom ops
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Added OPENVINO ci pipeline
Added new pipeline for openvino provider,
made changes to support the docker build and
onnxruntime build with openvino.
Signed-off-by: Luis Daniel Castellanos <luis.daniel.castellanos@intel.com>
* Enabled all unit tests for OpenVINO EP
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Fixed syntax issue in run_docker_build.sh file
* Added missing default OPENVINO_VERSION
Default value for OPENVINO_VERSION env was
missing causing the build to fail
* Added install Model Optimizer deps step
* Fixed python unit tests and some tests from onnx_backend_test_series
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Fixed indentation bug
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Disabled some of the python backend tests for OpenVINO
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Disabled some model tests
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Remove Duplicate checks for openvino in build.py
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Modified GetCapability for FP16
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Disabled GPU FP32 tests that are not supported
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Convert modelProto to string and use it in compile
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Pass byte-array input args to MO
* Serialized ModelProto passed in-memory to MO
ModelOptimizer python module receives the serialized ModelProto
in-memory.
Uses appropriate ONNX function to load the serialized bytes.
* Make Py_Finalize compatible with older python versions
Also, remove pFunc unassigned variable possibility.
* Fallback if input dims of Matmul is greater than 2
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* fixup: Device #define syntax
* Updated the documentation
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Enable dynamic dim value
* removed commented out code
* Added Dockerfile for openvino EP
Updated instructions on dockerfiles/README.md file
Signed-off-by: Luis Daniel Castellanos <luis.daniel.castellanos@intel.com>
* Disabled fp16_inception_v1 test
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Code formatting with clang-format
Uses style from the .clang-format file in root directory.
* fixup: docker tag and build error fixes
* Heuristics to automatically detect batching
Distributes slices from batch into parallel infer-request objects.
* Handle disabled tests in GetCapability
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Disabled average pool and max pool if ceil_mode is 1
Also dilations are not supported if they are greater than 1
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Disabled Unsqueeze int32 test
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* changes to fix output results bug
* Disabled a few C++ unit tests for MYRIAD FP16
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Manually revert '9fe162bb Enable dynamic dim value'
Reverts compile time setting of dynamic shape
Reverting manually due to significantly huge auto-revert conflicts.
* Fixed unused variable warning
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Disabled Mul test for GPU_FP16 due to accuracy issue
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* VPU documentation update
* Disabled inception_v1 for MYRIAD and HDDL
*Also disabled few C++ accuracy tests for HDDL
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* updates from upstream
* use the new CustomOpApis for I/O interfacing
* Pass initializers as subgraph meta-def inputs in GetCapability()
Requirement due to API changes introduced with PR# 1019.
* Remove obsolete functions
* Save indexes of graph inputs from fused_node info
Both inputs and initializers are passed as data inputs to the
infer function. To identify only inputs among them, save thier
index info from fused_node in Compile function.
* Documentation changes to enable VPU
* Fix VPU related changes in documentation
* Fix minor changes in documentation
* Fix VPU related changes in documentation
* Use Node.In/OutputDefs() to track graph inputs and outputs.
Don't use graph_viewer's GetInputs() or
GetInputsIncludingInitializers().
* Permit "SAME_UPPER" auto_pad attribute from MaxPool
* Disabled fp16_tiny_yolov2 in onnx model tests
* Updated documentation to include configuration guides for myriad and hddl
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Use 8 Infer requests only for VAD-R
* disable debug prints
* Clang-format source files
* Updated BUILD.md with OpenVINO R5 links
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Disabled same upper python tests
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Update test exclusion syntax
* Change path of install_onnx.sh
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Disable tiny_yolov2 in broken tests
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Revert "Change path of install_onnx.sh"
This reverts commit ba9db165f3be430f2aff1ef413299ed04637196a.
This change is only required for Intel internal CI pipeline until
the settings are matched with the upstream's CI pipeline.
* Added debug statements for debugging CI error
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Add --build_wheel to linux openvino pipeline
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Added -v option to onnx_test_runner for debugging
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Removed path change patch
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Added -c 1 to onnx_test_runner
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Refactor MO python invocation in separate function
Cleans up Model Optimizer python invocation check and conversion
logic. Invokes MO only once in GetCapability() and passes the
IR strings (xml and bin) to the Compiler as meta-def attributes.
* Add comments
* code cleanup and comments
* Code cleanup for GetCapability
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Removed unnecessary files
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Revert "Added -v option to onnx_test_runner for debugging"
This reverts commit d1dd70938a94d648df1a1dbbc2e48d0b97e49ec8.
* Revert "Added debug statements for debugging CI error"
This reverts commit b86d41afed2aa29c3508155d6f9c8d3a7263cc60.
* incorporate Status Code changes
* ComputeFunc returns Status::OK() on success
* Use test names to disable tests for MYRIAD and VAD-R
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Rename local identifiers from CNNNetwork to OpenVINO network
CNNNetwork is an OpenVINO's API class that represents more than
just convolutional neural networks (CNNs). Renaming helps to avoid
confusion that the API's only support CNN type models.
* Added error message if building on windows
* Removed duplicate option in Cmake
* Removed unnecessary parameters in activation_opt_test
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Refactor Map search and access logic for efficiently and cleanliness.
* use C++ style casts
* Use os.path.join for python directory path operations
* use C++ style casts
* EP classes should use onnxruntime namespace
* Clean up fixes from PR comments
* Don't explicitly shutdown Py interpreter
* Remove debug print statements
Prints will be re-enabled later with a logging mechanism with
debug/verbose printing options.
* Decrement ref counts for used pyObjects
* Restore build instructions for other compilers
Content under the "Using other compilers" section has been
accidentally deleted by a previous commit. Restoring back that
content from the latest upstream repo.
* CMake code cleanup
Code clean up, commenting and formatting of CMake code.
* Don't pass the unused device_info parameter to OpenVINOGraph ctor.
* Add support for multiple I/O data types
Adds support for the following tensor data types for graph inputs
and outputs:
1) float
2) float16
3) int32
4) int16
5) int8
6) uint16
7) uint8
* cleanup setup.py module list definition
* Deduce index of input using tracked input index map
Ignores initializers in case they are ordered before inputs.
* Removed debug statement in MO code
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* PR feedback
* Removed per_sample_tolerance for openvino
* Removed unnecessary disabled tests
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Removed debug function
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Disabled tiny_yolo_v2 due to accuracy issues
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Changed the disabled reason for broken tests
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Disabled Reshape with no input
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Python formatting with Autopep8
* Minor fix for MYRIAD devices
* Added zero dimension check
*Removed setting batch size for the network
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Set the threshold to larger value for MNIST
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Removed setting higher threshold in provider_test_utils
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Check for --use_openvino in python wheel setup.py
Add openvino modules to the setup script for building the wheel
package only for --use_openvino a build option.
* Removed nullptr checks for GetNode()
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
- Added Python script to post the code coverage data to the MySQL table used for dashboard
- Added a build job to run a windows cpu debug build on every merge on master, and run the script
- Removed the code coverage step from the CI build
* Initial check in
* Add win x86
* minor update to x86
* update win-ci
* update win-ci
* update win-x86ci
* add linux and mac templates
* add nuget pipelines and test templates
* remove buildConfig
* add compliance template
* fix minor typos
* update pool for macos
* update mac agent pool
* update macos pool
* update agent pools for tests
* turn off debug build for testing
* some modifications to packaging scripts
* change ordering of compliance tasks
* Add mklml pipeline
* Add packagename variable to mklml pipeline
* remove unrequired dependent jobs from mklml pipeline
* Update build command for macOS legs in mklml and cpu pipeline
* Set vcvars to true
* Add no contrib ops pipeline
* Add no-contrib-ops pipeline
* set vcvars to true for package tests
* remove repetition in nuget templates
* get buildarch correct
* get name of test template correct
* remove steps from test_all_os.yml
* add parameters to test_all_os.yml
* Need jobs, not steps
* set envars for disablecontrib ops
* add cleanup tasks and CG to package tests
* fix path to cleanup script for macos
* remove buildDirectory -- not needed
* remove fp16tiny_yolov2 model from nocontribops tests
* remove debugging info
* fix individual linux pipelines to use correct template
* remove unneeded bak_latest2
* increase timeout to 120 to allow for variance
* turn off code coverage report
Advance ONNX submodule to 5c51f0dbbe88ee1536f17ee7bd462b2ab3772c52
This commit in ONNX contains a fix to ConstantOfShape test data.
Uncomment ConstantOfShape.
Update test script, make sure exclusions are uniform.
* added the runcoverage powershell script
* updated the run coverage script. added installation to the windows CI for trying
* exclude other parts of win ci
* fix in the download script
* fix in the download script
* fix in the download script
* fix in the download script
* fix in the download script
* fix in the download script
* fix in the download script
* fix in the download script
* fix in the download script
* added the runtestcoverage script to the pipeline
* some typo fix
* formatting
* re-commenting previously commented block
* cleaned up the powershell script
* fix path in pipeline
* fix path in pipeline
* fixed model path
* some fixes
* excluded long running tests
* add the publish job
* uncomment other tasks
* fixed excluded tests
* some format correction
* stopped running the test debug
* try placing the tes-all at the beginning
* try running the failing test only
* edit run_coverage
* some fix
* skip onnx_model_test
* Added memory size log in powershell script
* try running the onnxruntime_test_all.exe separately from codecov
* enable error reporting, and double memory size in powershell
* corrected the set-item
* remove memory resize, since we are already at max 2 GB
* fixed the tvm.dll issue
* added back the onnx tests in codecov. added back the regular test run
* cleanup
* remove * from the the module path
* add junction target resolution for modules dir
* remove junction-resolution
* reduced tests
* added target extraction for the junction paths in build machine
* added the appropriate change in win ci pipeline to call the updated ps script
* fix typo
* added back all the tests that were disabled
* try fixing the source root
* cleanup and enable all tests
* increase timeout for windows CPU CI due to codecoverage
* templatized the code coverage steps. Conitnue on error with any codecoverage step
* change quote marks
* add version filter to failed tests
* exclude test from backend
* exclude shrink from opset 9
* fix compile err
* exclude certain version of constant shape
* enable flatten test
* fix compile err
* comment mvn test
* disable constantofshape test in x86
* disable x86 test
* get model version from imported opset
* test linux x86 case
* disable nonzero opset 10
* make mutex const
* test filter by commit id
* adjust substr offset
* Limit test platform
* remove change impacting TFModleInfo.h
* refactoring
* refactoring
* test x86 pipeline with filter
* add comment
* restrict version extraction on non-win
* restrict version extraction on non-win
* add tag
* exclude case from backend test
* remove dup
* remove dup
* make script runnable
* hard code adsolute path
* refactor log
* fix x86 compile err
* fix x86 compile err
* fix x86 compile err
* sync with latest tensorrt
* switch to regex
* fix cpu pipeline err
* test filter
* disable nonzero from all versions
* Add version and latest commit id to ORT Server
* Update cmake
* Change build id to build number
* Use target_compile_definitions instead of add_definitions
* add x86 legs to ci
* minor update
* update platform from x86 to Win32
* remove --use_mklml from x86 build
* add win x64 mklml pipeline
* remove pybind and use_tvm from win x86
* Add build pipelines for --disable_contrib_ops (mac, lnx, win)
* remove --gen-doc generation for x86
* set environment variables during build to disablecontribops=on
* update to match aiinfra pipelines
* update test data url
* update mac pipeline test data
* remove gen_doc from win x64 leg
* update model files for nocontribops
* reset win-ci-pipeline.yml
* remove confidential models
* Refactor C# to handle x86
* update run script
* Add Native win x86 tests
* Add native x86 tests for Linux
* Update linux tests scripts to control which tests are run
* update linux image name for x86 to prevent using cached image
* update to not run unit python unit tests unless pybind is specified
* remove --build_wheel as a core common arg. Python cannot run on x86 build
* update OrtGetNumOfDimensions to OrtGetDimensionsCount in rest of C#
* Finer control over when Python tests are run
* add --build_wheel to linux pipeline, instead of run_build.sh
* add --build_wheel to all ci configurations
* update per review comments
* Enable model tests for ORT Server in build script
* Nightly build pipeline definition
* Force prune docker images
* Clean up containers before prune images
* Update the ORT-SRV ci pipeline setup
* Update pip package installation for server tests
* Install requests package in build setup
* Check if python dependencies exists before install
* Accomodate missing optional 'axes' when 'steps' is present in Slice op (#946)
* Accomodate missing optional axes when steps is present in Slice implementation
* PR feedback
* Update package links (#937)
* Update package links
* Minor fix
* Update README.md
* Minor edit
* Update onnx commit (#949)
* Update onnx commit
* disable failing tests which don't have to be fixed for this release
* dummy change to fix file permission
* fix file permission
* add --gen_doc to ci_build
* make gen-doc conditional to build/test step
* some fix in the git diff check
* some more trick on doc diff
* updated for input/output
* updated the contrib operator doc
* fix on missing input output descriptions
* fixed the problem of missing doc string, due to protobuf optimization
* fix
* revert last change
* moved gen_doc.py to /tools/python
* fixed typo
* Simple integration into CMake build system
* Adds vcpkg as a submodule and updates build.py to install hosting dependencies
* Don't create vcpkg executable if already created
* Fixes how CMake finds toolchain file and quick changes to build.py
* Removes setting the CMAKE_TOOLCHAIN_FILE in build.py
* Adds Boost Beast echo server and Boost program_options
* Fixes spacing problem with program_options
* Adds Microsoft headers to all the beast server headers
* Removes CXX 14 from CMake file
* Adds TODO to create configuration class
* Run clang-format on main
* Better exception handling of program_options
* Remove vckpg submodule via ssh
* Add vcpkg as https
* Adds onnxruntime namespace to call classes
* Fixed places where namespaces were anonymous
* Adds a TODO to use the logger
* Moves all setting namespace shortnames outside of onnxruntime namespace
* Add onnxruntime session options to force app to link with it
* Set CMAKE_TOOLCHAIN_FILE in build.py
* Remove whitespace
* Adds initial ONNX Hosting tests (#5)
* Add initial test which is failing linking with no main
* Adds test_main to get hosting tests working
* Deletes useless add_executable line
* Merge changes from upstream
* Enable CI build in Vienna environment
* make hosting_run*.sh executable
* Add boost path in unittest
* Add boost to TEST_INC_DIR
* Add component detection task in ci yaml
* Get tests and hosting to compile with re2 (#7)
* Add finding boost packages before using it in unit tests
* Add predict.proto and build
* Ignore unused parameters in generated code
* Removes std::regex in favor of re2 (#8)
* Removes std::regex in favor of re2
* Adds back find_package in unit tests and fixes regexes
* Adds more negative test cases
* Adding more protos
* Fix google protobuf file path in the cmake file
* Ignore unused parameters for pb generated code
* Updates onnx submodule (#10)
* Remove duplicated lib in link
* Follow Google style guide (#11)
* Google style names
* Adds more
* Adds an additional namespace
* Fixes header guards to match filepaths
* Consume protobuf
* Unit Test setup
* Json deserialization simple test cases
* Split hosting app to lib and exe for testability
* Add more cases
* Clean up
* Add more comments
* Update namespace and format the cmake files
* Update cmake/external/onnx to checkout 1ec81bc6d49ccae23cd7801515feaadd13082903
* Separate h and cc in http folder
* Clean up hosting application cmake file
* Enable logging and proper initialize the session
* Update const position for GetSession()
* Take latest onnx and onnx-tensorrt
* Creates configuration header file for program_options (#15)
* Sets up PredictRequest callback (#16)
* Init version, porting from prototype, e2e works
* More executor implementation
* Adds function on application startup (#17)
* Attempts to pass HostingEnvironment as a shared_ptr
* Removes logging and environment from all http classes
* Passes http details to OnStart function
* Using full protobuf for hosting app build
* MLValue2TensorProto
* Revert back changes in inference_session.cc
* Refactor logger access and predict handler
* Create an error handling callback (#19)
* Creates error callback
* Logs error and returns back as JSON
* Catches exceptions in user functions
* Refactor executor and add some test cases
* Fix build warning
* Add onnx as a dependency and in includes to hosting app (#20)
* Converter for specific types and more UTs
* More unit tests
* Update onnx submodule
* Fix string data test
* Clean up code
* Cleanup code
* Refactor logging to use unique id per request and take logging level from user (#21)
* Removes capturing env by reference in main
* Uses uuid for logging ids
* Take logging_level as a program argument
* Pass logging_level to default_logging_manager
* Change name of logger to HostingApp
* Log if request id is null
* Update GetHttpStatusCode signature
* Fix random result issue and camel-case names
* Rollback accidentally changed pybin_state.cc
* Rollback pybind_state.cc
* Generate protobuf status from onnxruntime status
* Fix function name in error message
* Clean up comments
* Support protobuf byte array as input
* Refactor predict handler and add unit tests
* Add one more test
* update cmake/external/onnx
* Accept more protobuf MIME types
* Update onnx-tensorrt
* Add build instruction and usage doc
* Address PR comments
* Install g++-7 in the Ubuntu 16.04 build image for vcpkg
* Fix onnx-tensorrt version
* Check return value during initialization
* Fix infinite loop when http port is in use (#29)
* Simplify Executor.cc by breaking up Run method (#27)
* Move request id to Executor constructor
* Refactor the logger to respect user verbosity level
* Use Arena allocator instead of device
* Creates initial executor tests
* Merge upstream master (#31)
* Remove all possible shared_ptrs (#30)
* Changes GetLogger to unique_ptr
* Reserve BFloat raw data vector size
* Change HostingEnvironment to being passed by lvalue and rvalue references
* Change routes to getting passed by const references
* Enable full protobuf if building hosting (#32)
* Building hosting application no longer needs use_full_protobuf flag
* Improve hosting application docs
* Move server core into separate folder (#34)
* Turn hosting project off by default (#38)
* Remove vcpkg as a submodule and download/install Boost from source (#39)
* Remove vcpkg
* Use CMake script to download and build Boost as part of the project
* Remove std::move for const references
* Remove error_code.proto
* Change wording of executable help description
* Better GenerateProtobufStatus description
* Remove error_code protobuf from CMake files
* Use all outputs if no filter is given
* Pass MLValue by const reference in MLValueToTensorProto
* Rename variables to argc and argv
* Revert "Use all outputs if no filter is given"
This reverts commit 7554190ab8e50ba6947648c2f3e2a3d4d9606ce0.
* Remove all header guards in favor of #pragma once
* Reserve size for output vector and optimize for-loop
* Use static libs by default for Boost
* Improves documentation for GenerateResponseInJson function
* Start Result enum at 0 instead of 1
* Remove g++ from Ubuntu's install.sh
* Update cmake files
* Give explanation for Result enum type
* Remove all program options shortcuts except for -h
* Add comments for predict.proto
* Fix JSON for error codes
* Add notice on hosting application docs that it's in beta
* Change HostingEnvironment back to a shared_ptr
* Handle empty output_filter field
* Fix build break
* Refactor unit tests location and groups
* First end-to-end test
* Add missing log
* Missing req id and client req id in error response
* Add one test case to validate failed resp header
* Add build flag for hosting app end to end tests
* Update pipeline setup to run e2e test for CI build
* Model Zoo data preparation and tests
* Add protobuf tests
* Remove mention of needing g++-7 in BUILD.md
* Make GetAppLogger const
* Make using_raw_data_ match the styling of other fields
* Avoid copy of strings when initializing model
* Escape JSON strings correctly for error messages (#44)
* Escape JSON strings correctly
* Add test examples with lots of carriage returns
* Add result validation
* Remove temporary path
* Optimize model zoo test execution
* Improve reliability of test cases
* Generate _pb2.py during the build time
* README for integration tests
* Pass environment by pointer instead of shared_ptr to executor (#49)
* More Integration tests
* Remove generated files
* Make session private and use a getter instead (#53)
* logging_level to log_level for CLI
* Single model prediction shortcut
* Health endpoint
* Integration tests
* Rename to onnxruntime server
* Build ONNX Server application on Windows (#57)
* Gets Boost compiling on Windows
* Fix integer conversion and comparison problems
* Use size_t in converter_tests instead of int
* Fix hosting integration tests on Windows
* Removes checks for port because it's an unsigned short
* Fixes comparison between signed and unsigned data types
* Pip install protobuf and numpy
* Missing test data from the rename change
* Fix server app path (#58)
* Pass shared_ptr by const reference to avoid ref count increase (#59)
* Download test model during test setup
* Make download into test_util
* Rename ci pipeline for onnx runtime server
* Support up to 10MiB http request (#61)
* Changes minimum request size to 10MB to support all models in ONNX Model Zoo
* Adding versioned dlls to tar/zip packages
* fix syntax error
* fix version name of dylib
* minor fix in the target
* update pattern for versioned dylib files
* move files
* move files
* Remove NonMaxSuppression from Contrib op, move it to Onnx domain, opset 10
* move NMS out of namespace contrib
* update data type in UT
* update to latest onnx
* white list the node test for Mod which is not implemented yet
* enable android build
* Add 'log' to onnxruntime_EXTERNAL_LIBRARIES
* Remove cmake about header_files_test.cc
* Add Android CI pipeline
* Remove some ms-specific(?) ci
* Fix bash error
* Add execute flag for install_deps_android.sh
* Add install_ubuntu_for_android.sh
* Remove python in deps for android
* Add comment for BUILD_ARCH
* Set BUILD_SERVICE to cpu
* Set BUILD_OS in run_build.sh
* Fix -o bug in run_build.sh
* Android -> android
* Correct the android ndk location
* Checkout submodules in my own azure pipelines
* Revert "Remove some ms-specific(?) ci"
This reverts commit 302463213480487d8944c3127a3b311c591d55c0.
* Revert "Checkout submodules in my own azure pipelines"
This reverts commit 1acfb6755f933e532b8312ca35bb4900a833903f.
* Add docker image clean script
* Change the command not to generate warning if no such image presents
* Update linux-gpu-ci-pipeline.yml
* Update linux-ci-pipeline.yml
* Update azure-pipelines-py-packaging.yml
* Exclude unreferenced global data and op doc strings in the opschema object. The first causes a decrease in the binary size by at least 85k. The latter reduces resident memory size.
* Update onnx to incorporate my PR that fixes SetDoc compiler warnings
* Ensure Linux binaries are built with debug info. Extract debug info out of the main binaries. Strip the main binaries.
* add binutils
* add uname
* add binutils
* remove linux portion
* Update onnx
* Support updated function schema in ORT
* Update onnx related commit hash
* Check out an older commit in ONNX
* Add support for subgraph attribute
* Add comments
* Test protobuf-lite
* Test protobuf-lite
* Test protobuf-lite
* Optimize protobuf usage for LITE_RUNTIME to reduce the binary size of
onnxruntime.dll. More details can be found here https://developers.google.com/protocol-buffers/docs/proto.
The reduction is significant. For commit id: 4873b452151bafe49da332aaeab639ef0318fc1ca28d728, the size
reduced by ~700K; from 4873728 to 4172800.
* Add LITE_RUNTIME flag in in.proto files
* Fix merge conflict.
* Address PR comments
* Forgot to add 2 files + fix linux and gpu build errors.
* Fix build errors + test failures
* Fix cuda tests
* Fix tensor rt build
* Use full protobuf for trt
* Address PR comments
* Print tensor shape proto as text string for easier debugging
* updated cmake files for trt
* added trt execution provider
* added trt basic test
* removed trt_path action attribute
* Add files via upload
* Update build.py
* Update trt_allocator.h
* fixed issues found by reviewers
* changed cast operator
* added comment for custom kernel implementation
* changed auto to auto&
* changed to function compile APIs for TRT execution provider
* changed to function compile APIs for TRT execution provider
* added new DType DInt64
* adapted to the changes of onnxruntime_c_api
* removed trt kernel (use function compile instead)
* updated onnx-tensorrt submodule
* set default memory type to TRT fused kernel
* resolve merge conflict
* fixed the issue that USE_CUDA conflicts with USE_TRT
* construct graph by adding nodes in topological order
* made changes for Windows
* change buffers type
* bypass HasImplementationOf check for TRT XP because TRT kernel is not registered
* added domain to version info in rebuilt model proto
* added trt to test option list
* added DomainToVersionMap() to GraphViewer
* removed Copy()
* fixed broken code
* format the code to clang format
* used local reference to the frequently used values
* fixed a couple of issues according to reviewers feedback
* fixed a couple of issues according to reviewers feedback
* added python binding for TRT and enable use_cuda when use_trt is on
* fixed a redefinition issue
* changed shared_ptr to unique_ptr on trt engines, and made a few changes required by reviewers
* enabled trtexecution provider for unit tests
* renamed trt to tensorrt
* added tesorrt to python binding
* update submodule onnx and onnx-tensorrt
* made a couple of minor changes based on reviewer's feedback
* added CUDA_CHECK
* removed test code
* fixed broken code after merge
* updated onnx-tensorrt submodule
* added post processing to align trt inputs/outputs with graph inputs/outputs
* updated onnx submodule
* added CUDA fallback for TensorRT and fixed TensorRT cmake issue
* added ci pipeline for tensorrt and removed some redundent code from trt xp
* fixed syntax issue
* updated onnx-tensorrt submodule
* fix trt build problem by: (#602)
1. Add additional /wd for debug build
2. Add io.h for additional targets
3. Bring back mb version of getopt
* Update install_ubuntu.sh
* Update linux-gpu-tensorrt-ci-pipeline.yml
* Update linux-gpu-tensorrt-ci-pipeline.yml
* Update run_build.sh
* Update run_build.sh
* Update run_build.sh
* Update run_build.sh
* fixed the issue that GetKernelRegistry returns nullptr
* merged master to this branch
* moved some data types to private
* fixed tensorrt CI pipeline issue
* customized test data for TensorRT pipeline
* added onnx-tensorrt in json file and fixed an issue in ci script
* added comments
* sync onnx and maintain old version history for removed exp ops in onnx runtime.
* update
* updating to specific onnx commit - remove exp ops.
* update
* disable the 3 failures to push the change as it's blocking folks.
* update test
* add variables for version number and git commit hash
* fix typo
* fix typo
* some logging
* some logging
* some logging
* some logging
* some logging
* some logging
* some logging
* some logging
* some more edits to see generic scripts can print
* working
* fixing windows git hash
* try quoted echo
* fix git rev-parse
* echo without quotes
* removed commit hash from artifact filename, added long commit hash as a file inside
* added the missing commit id parameter
* fix windows pipeline
* keep only win 64, others disabled
* remove disabling conditions
* cross compile x86 linux
* fix comments
* install multilib for ubuntu cross compile
* remove tailing slash
* fix -fPIC relocations for x86 target too
* add asm make flag
* fix x86 compile err
* test x86 with zlib and png
* Disable zlib from x86
* install x86 python header
* remove cross-compiling changes
* test 32bit ubuntu
* add x86 ubuntu docker file
* add x86 as arch parametr for docker build
* config pipeline
* avoid dotnet install
* install cmake
* skip dep install
* use latest ubuntu
* install latest cmake
* install x86 deps
* configure cmake
* install ninja
* correct ninja dir
* apt get re2c
* install onnx
* set processor x86
* disable warning
* skip test
* disable test
* disable test
* find lib
* fix typo
* restore test
* disable backend model test
* disable test
* fix test err
* stop installing onnx
* disable onnx test on x86
* restore yml
* mergef with master yml
* cancel needless config setting
* enable x86 flag
* restore all onnx tests
* fix yml typo
* install onnx
* add back x86 flag
* disable cases
* disable case
* disable cases
* add macro to disable cases
* fix typo
* print platform
* remove condition
* Initial commit
* More changes
* More changes
* More changes
* More changes
* PR feedback
* Commiting Azure build config file
* Fix build pipeline
* Cleanup build dir template addition
* Remove conda modules download step
* PR feedback
* Revert x86 arguments to as they are currently
* More changes
* use CUDA 9.1 for both linux and windows
* added powershell scripts for cuda props setup/cleanup
* fix yml syntax
* set path to cuda9.1 bin
* correct label
* ad --cuda_version
* added some log to browse the directory
* disabled jobs other than win gpu to save some resource while testing
* add msvc_toolset=14.11
* added more logs
* log the props file
* remove setting vcvarsall
* try some modificationi n build.py
* fix typo
* let the config Step modify envoronment
* set some more env vars manually
* try reordering vcvars after cuda props copying
* use single script for build and test
* single line script
* remove extra quote
* cleanup trial changes
* added linux packaging template and pipeline
* Update linux-packaging-pipeline.yml for Azure Pipelines
* fix path seperator
* update copy command for linux
* fixed linux gpu artifact name, added mac build
* fixed linux gpu artifact name, added mac build
* fixed vmImage syntax
* use 1 model at a time for macos
* added onnx test on Mac CI
* some refactor of the pipeline scripts
* try fixing the tensorproto for x86 build
* try __cdecl
* try C-style cast
* use ORTAPICALL
* put the deleter under the namespace
* added packaging pipeline
* Update win-ci-pipeline.yml for Azure Pipelines
* Update win-ci-pipeline.yml for Azure Pipelines
* Update win-ci-pipeline.yml for Azure Pipelines
* Update win-ci-pipeline.yml for Azure Pipelines
* Update win-ci-pipeline.yml for Azure Pipelines
* Update win-ci-pipeline.yml for Azure Pipelines
* Update win-ci-pipeline.yml for Azure Pipelines
* Update win-ci-pipeline.yml for Azure Pipelines
* put the c-api header file at root instead of under core/session
* Update win-ci-pipeline.yml for Azure Pipelines
* Update win-ci-pipeline.yml for Azure Pipelines
* Update win-ci-pipeline.yml for Azure Pipelines
* parameterize the windows build script
* Update win-package-pipeline.yml for Azure Pipelines
* fixed indenting
* fixed indenting
* fix parameter reference syntax
* try using arch = amd64 for the vcvarsall
* remove duplicate tasks
* use vcvarsall
* some more refactor
* fix typo
* fix typo
* factored out the packaging step into a template
* add x86 build to package pipeline
* use amd64 for vcvars arg
* added gpu pipeline. added msbuild platform param
* fix the msbuild platform
* use amd64 host for x86 build
* use buildarch=x86 for vcvarsall
* remove vcvars from setup steps
* add some logging for PNG lib, and disable fns_candy demo for win32
* set allocator alignment to 32 bit for win32 compiler
* disable parallel execution test for x86
* use 64 bit toolchain for x86 build
* add missing -T flag for toolset
* fix string delimietr in workingdirectory name for package build test step
* fix gpu pipeline
* make io_types test conditional
* use cuda 10 instead of cuda 9.1, similar to the ci build
* try some workaround on the io test
* undo inadvertent local change in build.py, also reenable the io test
* make all test run single threaded
* blacklist few failing tests for x86
* added some log in build.py
* edit build.py to disable parallel test
* add the failed tests into the blacklist for win32
* add tf_pasnet_large to blacklist
* change control flow for build.py onnx tests
* add README, license and TPN to the package
* updated build.py test sequence for parallel executor
* updated onnx test flow as per review comment
* add type checking log in the compare_mlvalue
* fix type cast
* blacklist some failed test as of now
* one more blacklisted test
* update packaging numpy version to 1.15.0
* update version in numpy version in linux
* Install numpy 1.15.0
* Finish up numpy requirement after test
* Try fix
* Fix ci script
* support non-tensor types
* support non-tensor types.
* support non-tensor types.
* fix compilation issues
* fix compilation issues
* Build without mkldnn for release packages. We'll default to MLAS.
* Remove tvm as well
* Add openmp
* Update cast kernel to support to/from string
* Update namespace
* Add support for literal numeric case
* Update to support -INF test
* Update kernel registration for cast
* Update ONNX to 1.4.1
* Update registy api
* Resolve some comments
* Update cast kernel implementation
* Resolve comments
* Fixed test data in onnx
* Update cast kernel implementation
* Resolve PR comments
* Update cast_op.cc
* Update onnx commits info
* Update comments
* try removing the --version-script
* remove --no-undefined flag
* remove the -rpath linker flag
* remove the -rpath linker flag, including the -Wl
* remove the --whole-archive flags
* added -all_load -noall_load flags in place of --whole-archive and --no-whole-archive
* spell correct all-load
* set the MacOS specific cmake configs with if(APPLE) condition
* added --build_shared_lib to mac CI
* Updated TPN
* Update batch_norm_op_test.cc
* Update ThirdPartyNotices.txt
* Update ThirdPartyNotices.txt
* Update readme with package links
* Update README.md
* Update README.md
* Update README.md
* Merged Ryan and TPN changes into single PR
* minor fix
* added mkldnn to GPU pipeline. Required by C# library as it is the default execution provider
* fixed typo in runtest.sh
* some fixes
* some fixes
* some fixes in the runtest.sh
* added test data url
* fixes on the dotnet test scripts
* fix on prior mistake regarding installation of apt-transport-https
* added verbosity in the test run for easy debugging
* updated comment in the runtest.sh
* Update ONNX version to pickup Scan spec change that adds scan_output_axes.
Add logic to transpose an output
- write to temporary buffer when executing subgraph
- transpose temporary buffer into Scan output when execution completes
Add unit tests
* Update to ONNX dbf3581835e3a05716e10587511d7ab3b2cdc386 to pickup inferencing bugfix.
Update test to match.
* Disable some tests for opset 9 operators that haven't been implemented yet.
* Added test data arguments to build.py, modified win-ci-pipeline build.
* Updated CI builds to use template tasks, added test data args, removed AZURE_BLOB_KEY uses.
* Fixed up set test data step template.
* Initial checking for CSharp GPU support
* Enabled C# for GPU build
* Update Onnxruntime to Ort
* Add runtime check for cuda dlls for windows
* Update pretrained model test, for models where name!=model.onnx
* lowered tolerance for float checks to pass new models
* ignore extra ._resnet34v2.onnx file in pretrained test
* Added tests for C# types
* added multithread c# test
* Added pretrained tests
* Added function to load data from PB file
* added exec task for protoc
* Upgrade gpu build to CUDA 10 + cudnn 7.3
* update the yaml file for python package building
* switch to the cuda9.1 docker file if the CUDA_VER is cuda9.1-cudnn7.1
* Add build step to remove the cuda msbuildcutomization file after build, otherwise, the cuda high version could impact the lower version build
* update vs path
* update the path
* Imlpement StringNormalizer
Add mixed language tests, test case insentive path.
* Create a locale on the fly. Default locale does not seem to create well.
* Add CI language-pack-en to make default locale available.
Catch and translate locale creation exception to make the message
meaningful.
* Make sure locales are configured on Ubuntu.
* Add pipeline for building python wheels for Windows/Linux CPU and GPU
* try enable mkldnn
* remove mklml
* Update python packaging configuration
* Try macos pywheel packaging
* Try removing mkldnn from mac build
* Use conda in mac agents
* Change to build release only
* Add the mac wheel packaging list to the packaging yaml
* Add mkldnn into mac wheels
* Add pipeline for building python wheels for Windows/Linux CPU and GPU
* try enable mkldnn
* remove mklml
* Update python packaging configuration
* Add python3.7 support
* Revert to disable the py37 packaging on windows
