* build e2e cppwinrt tests
* add use nuget task
* make all referenced to package version prop/target-ified
* remove dupe props/targets reference
* work around project.assets.json error by deleting it
* powershell test invocation
* switch to batch script
* print debug info
* update x86->x64
* stdio.h
* pushd/popd
* add csharp tests
* package.config -> packages.config
* typo
* x86 -> anycpu
* debug is default
* add test path
* update csproj as well
* debug
* really replace all package versions
* debug output
* really use [PackageVersion]
* sleep intead of converting async operation to task and waiting
* dont close software bitmap
* switch to powershell script
* remove binding check
* continue on failure
* continuse on error action
* continueOnError and errorActionPreference
* tabbing
Co-authored-by: Sheil Kumar <sheilk@microsoft.com>
* Change NNAPI CI to run on new NNAPI EP
* update android ci to mac 10.15 and remove in install cmake
* update the android ci to targe android api level 29
* remove unnecessary ndk install git submodule call
1. Increase job timeout, while we are investigating why the tests take much longer
2. Upgrade the linux docker image to manylinux2010, by request from Tianlei. (We had an offline discussion with Pranav and Tracy)
3. Remove the installation of "devtoolset-7" in the CUDA image. It was added for CUDA 10.0, it is not needed for CUDA 10.1. We have moved to CUDA 10.1.
* Add build option to disable traditional ML ops from the binary.
* Fix python tests by splitting tests for ML ops to a separate file. Exclude ML tests from onnx_test_runner and C# tests. Exclude ML op sources.
* Update Edge pkg pipelines with new MLops env variable and fix C# packaging pipeline tests to skip ML ops.
Modify gradle build so artifactID has _gpu for GPU builds.
Pass USE_CUDA flag on CUDA build
Adjust publishing pipelines to extract POM from a correct path.
Co-Authored-By: @Craigacp
1. Enlarge the read buffer size further, so that our code can run even faster. TODO: need apply the similar changes to python some other language bindings.
2. Add coreml_VGG16_ImageNet to the test exclusion set of x86_32. It is not a new model but previously we didn't run the test against x86_32.
* try mac pipeline
* fix path separator
* copy prebuilds folder
* split esrp yaml for win/mac
* disable mac signing temporarily
* add linux
* fix indent
* add nodetool in linux
* add nodetool in win-ci-2019
* replace linux build by custom docker scripts
* use manylinux as node 12.16 not working on centos6
* try ubuntu
* loosen timeout for test case - multiple runs calls
1. Fix the nuget cpu pipeline and put code coverage pipeline back.
2. Reduce onnx_test_runner's default logging level from WARNING to ERROR. Because there are too many log messages now.
3. Enlarge the protobuf read buffer size for onnx_test_runner. It was missed from PR #4020.
- Add support for ENABLE_LANGUAGE_INTEROP_OPS in training build which is enabled for nightly builds
- Fix passing of environment variables to `sudo docker run` in build definitions
- Fix setup.py package naming logic
* Add flake8 to Win CI build so it's re-enabled. It was in the static analysis build that is currently disabled so checks are not running.
Fix build.py to be compliant again.
Add prefix to flake8 output so it's (hopefully) easier to identify the errors in build output.
* Add to all builds in Windows CPU CI so they all fail quickly if there's an issue.
Add transformer glue test example to show how to use ORTTrainer to fine-tune a transformer model
Co-authored-by: liqun <liqun@OrtTrainingDev4.af05slrtruoetgaxwwjv5nsq5e.px.internal.cloudapp.net>
In this PR, we
1. create some APIs for creating NVTX objects
2. apply those APIs in pipeline-related operators and sequential executor.
As a result, we can explicitly see how a pipeline schedule is run by GPUs in
Nvidia's visual profiler. Note that these APIs are Linux only due to Nvidia's
limited support.
* Remove 'model_.' prefix for onnx model initializers in training
* fix test case remove redundant device test
* rename
* Fix state_dict/load_state_dict with frozen_weight
* nit
* Add monkey patch for pt opset 10
* remove pt patch in CI
* nit: newline
Change training perf test build to use "docker" instead of "sudo docker". The training perf test build runs in an environment that supports calling "docker" and not "sudo docker".
* gpt2 training perf
* gpt2 training perf
* debug
* debug
* debug
* fix bug
* minor
* on comments
* dynamic sql
* fix build
* minor
* linked hash
* on comments
* minor
* mem
* minor
Co-authored-by: Ethan Tao <ettao@microsoft.com>
Update install_deps.sh to use relative path from script directory to symbolic_opset10.py. This allows install_deps.sh to be called from different working directories.
* [java] - adding a cuda enabled test.
* Adding --build_java to the windows gpu ci pipeline.
* Removing a stray line from the unit tests that always enabled CUDA for Java.
* Enable running PEP8 checks via flake8 as part of the build if flake8 is installed.
Update scripts in \tools and \onnxruntime\python. Excluding \onnxruntime\python\tools which needs a lot more work to be PEP8 compliant. Also excluding orttraining\tools for the same reason.
Install flake8 as part of the static_analysis build task in the Win-CPU CI so the checks are run in one CI build.
Update coding standards doc.
* Added aarch64 build pipeline
* Fix build error
* Remove auditwheel repair which doesn't work with cross compiling
* Statically link C++
* Added auditwheel repair back and fix stdlib.h
* Remove extra space
* Add signed nuget package to publish ort-nightly nuget feed
* Push managed nuget as well
* Indentation fix
* Indentation fix
* Update gpu.yml to also publish directml nuget
* Fix typo in naming of task
* Fix C# log APIs. Fixes github issue #3409.
* Fix build error due to accidental duplication of GraphOptimizationLevel
* Fix runoptions
* Fix broken test. Add --blame switch to dotnet test cmd line to print the failed test in case of crash.
* initial change to transformer.py
* prepare e2e transformer tests
* refactor transformer tests
* put test python files in a flat folder
* fix typo pip install transform(s)
* python 3.6
* python version to 3.6 in install_ubuntu.sh
* remove argparser
* to use opset ver 12
* workaround loss_scale naming patch in case of loss_fn_
* assign self.loss_fn_ so it can be checked
* skip a few un-needed post-process steps
* fix loss_scale_input_name, clean up post process steps
* skip non-frontend tests
* move cpu/cuda related files to coresponding cpu/cuda folder (#3668)
Co-authored-by: Weixing Zhang <wezhan@microsoft.com>
* type cast for ratio is not necessary for dropout (#3682)
Co-authored-by: Weixing Zhang <wezhan@microsoft.com>
* thrustallocator is not needed since cub is used directly for gather now. (#3683)
Co-authored-by: Weixing Zhang <wezhan@microsoft.com>
* GatherND-12 Implementation (#3645)
* Renamed, UT passing
* Move GatherND CUDA Kerenl into onnxruntime
* Merge GatherNDOpTest
* Refactor Test code
* Merge CPU Kernel Impl
* Handle Negative Indice, Fix UT
* Improve CUDA kernel to handle negative index
* Minor Fixes
* Preserve GatherND-1 Cuda kernel
* Fix Mac build
* fix UT
* Fix Build
* fix GatherNDOpTest.double > CUDA error cudaErrorInvalidDeviceFunction:invalid device function
Co-authored-by: Sherlock Huang <bahuang@OrtTrainingDev3.af05slrtruoetgaxwwjv5nsq5e.px.internal.cloudapp.net>
Co-authored-by: Peng Wang (pengwa) <pengwa@microsoft.com>
* update with reviewers' comments
* testBertTrainingGradientAccumulation was not using rtol and may fail occasionally with small (e-06) difference
* fix merge mistakes
Co-authored-by: liqun <liqun@OrtTrainingDev4.af05slrtruoetgaxwwjv5nsq5e.px.internal.cloudapp.net>
Co-authored-by: Weixing Zhang <weixingzhang@users.noreply.github.com>
Co-authored-by: Weixing Zhang <wezhan@microsoft.com>
Co-authored-by: Sherlock <baihan.huang@gmail.com>
Co-authored-by: Sherlock Huang <bahuang@OrtTrainingDev3.af05slrtruoetgaxwwjv5nsq5e.px.internal.cloudapp.net>
Co-authored-by: Peng Wang (pengwa) <pengwa@microsoft.com>
The flags "--enable_wcos --use_winml" don't work with the latest VC++ and CMake. I don't know which caused the failure. But it doesn't work. Remove it to make the pipelines work first.
Will add them back before 1.3 release.
* add windowsai.yml for new Microsoft.AI.MachineLearning nuget
* temporarily add windowsai.yml to gpu.yml
* pass in build arch
* remove install onnx task
* no dml for arm or arm64
* refactor nuget pipeline defs
* update package creation
* pass in build and sources path
* missing hyphens
* copy license file
* fix parameter variable
* disable arm builds for now
* remove commented script block
* download pipeline atifcat name update
* set working dir
* Add bundling nuget script
* path combine
* null path
* combine needs parentheses
* binplace microsoft.* dlls in new nuget package
* update artifact name
* move merged nuget to artifacts directory
* move to merged subfolder in artifacts staging dir
* forward slash to back
* enable arm
* vcvarsall needs x64 vars setup
* Run Tests
* fix tests
* move global variables
* update yml to not have global variable in template
* removed parameters
* fixes
* Add build arch as an env variable
* ne not neq
* %Var% for batch script
* dont pass argument for x64
* disable arm tests
* skip csharp/cxx tests for microsoft nuget package
* remove test-win as it tests only c# cxx and capi
* test build for store apps
* dont build for store
* tools/nuget/generate_nuspec_for_native_nuget.py
* remove args.
* add new props and targets for microsoft.ai
* make windowsai props/targets static
* add dependency
* dont ship dot net props
* Remove c# fom windowsai nuget
* copy license file
* native packages must have win10 as the platform, not win
* cuda header in wrong if branch
* no dml for arm builds
* only build dml for x64/ x86
* User/sheilk/props update (#3616)
* prelim store work
* props
* Fix desktop nuget props/targets
* clean up targets and make store apps work
Co-authored-by: Sheil Kumar <sheilk@microsoft.com>
* update windowsai.yml with latest
* remove extra dloadhelpers
* Add abi headers to abi dir, and reference native includes
* update windowsai.yml
* minor update
* remove parameters
* add doesrp param
* hard code esrp to true
* add directml for x86/x64
* revert gpu yml changes
* add store builds
* add store builds
* add checks again in old way
* dup job names for store and desktop builds
* move all of the runtime binaries to win10 folder
* only set safeseh on x86
* disable the store builds for now... missing msvcprt.lib
* copy paste deletion...
* switch back to win- (#3646)
Co-authored-by: Sheil Kumar <sheilk@microsoft.com>
* use stahlworks
* & not supported in ado
* add cuda to cpu nuget(???) and EnableDelayedExpansion to enable x86 dml package
* revert nocontribops
* add underscore...
* extra win/win10 change
* merged nuget... still not being bundled...
* files in merged directory
* missing parens causing dml to be included in cpu package
* more diagnostic info
* switch dir to get-childitem
* wait for compression to complete
* add winml_adapter to mkml and gpu packages
* enable_wcos
* add mklml binaries
* props and targets missing from mklml
Co-authored-by: Sheil Kumar <sheilk@microsoft.com>
* Added FP16 transformations
* Revert "Added CMAKE_BUILD_TYPE to make building dynamic"
This reverts commit d3e17af1af655cfdc4d2fec33f52055caa525e85.
* Added FP16 transformations for FP16 builds
* Backend logic cleanup
Cleans the backend(intel_graph.*) code in the following ways:-
1. Minimize global usage: Since all the IR graphs need to be
re-generated on every Infer, it is bad practice to rely on globals
for their saving and usage as there would be multiple readers and
writers to the same global variable leading to incorrect usages or
contentions. This change replaces globals with locals where possible.
This change also fixes an existing bug with due to
incorrect global usage.
2. Remove all unused functions.
3. Remove all unused headers and prepocessor directives.
* removed commented out code
* Disabled default optimization for Intel EP
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Fix missed plugins.xml for python bindings
* Fixed the build after latest master changes
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Disabled unsupported ops for accelerators
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Added some more disabled ops
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Added environment variable to enable debugging
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Added more debug statements
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Fixed unsupported ops list for GPU and VPU
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Fixed unsqueeze unit tests
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Added error message to the status
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Overwrite Model proto with shape info from data
Overwrites the shape info of Model proto with the shape from
actual input data. Needed for inferring models with Dynamic
shapes.
* Removed print statement and disabled where op
Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
* Disabled Reshape with Empty initializer
* Added more debug statements for 1P
* Don't allow 1D inputs with symbol for dimension
* Disabled some 3rd phase ops
* Disabled split and added zero dimension check for OutputDefs
* Cleanup zero dimensionality check
* Added different data type check for inputs and initializers
* Added conditions for Mod, Cast and Pad
* Removed unused variable
* Disabled scan and added conditions for squeeze
* Added changes for fixing all C++ unit tests
* Implements Backend Manager class for caching
Backend Manager provides a layer of indirection between EP interface
and OV backend that provides caching services for models with
symbolic dims in input shapes.
* clean up commented blocks
* clang-formatting
* Read I/O type info from ModleProto
Read the tensor element type information from ModelProto object,
as FusedNode is no longer available.
* code cleanup
* clang-formatting
* Added print statement for jenkins
* Disabled some python tests
* Changed the path of convert fp32 to fp16 hpp
* Added conditions for BatchNorm in GetCapability
* Fixed failed tests
* Revert "Added conditions for BatchNorm in GetCapability"
This reverts commit c3c28c3b00d27892c42546b35dacdd807a48ee90.
* Added Intel to onnxruntime backends
* pick up vars set by OV package setupvars.sh
* Added conditions for Identity
* remove a few cout prints
* Added conditions for GPU_FP32 unit tests
* Revert "pick up vars set by OV package setupvars.sh"
This reverts commit 8199e029c03eae21a1a7ef6bfdc93d00e5d0198b.
* Commented out fatal message for protobuf
* Might need to be removed
* Add interface class for current backend
* moved common logic to base class
* simplified cpu backend
* Removed unused headers
* use vectors to save i/o tensors for windows compatibility
* move utils fxns to backend_utils namespace
* rename ov_backend to ibackend
* Factory pattern for backend creation
* rename CPU backend to Basic backend
* renamed to vad-M and added to factory list
* Added conditions for VPU
* Added print statements
* Changed the logic for checking for symbolic shapes
* Modified logic for zero dimension check
* Removed VPU single dimension condition
* Removed comments
* Modified logic in DimensionCheck method
* Remove legacy OpenVINO EP
Remove all the legacy code for OpenVINO EP. UEP code will take its
place going forward.
This change does NOT remove OVEP files in the following areas asa
they will be reused by UEP:-
1. Documentation: All .md files
2. Docker releated files
3. Python bindings
4. Java bindings
5. C# bindings
6. ORT Server
7. CI pipeline setup files
* Rename Intel EP to OpenVINO EP
* Added unique names to the subgraphs
* Removed subgraphs with only constant inputs
* Modified subgraph partitioning algorithm to remove const input subgraphs
* Apply suggestion to onnxruntime/core/providers/openvino/openvino_execution_provider.cc
* Tracking output names to fix the output order bug
* Changed output names to a unordered map
* Modified logic to check for symbolic input shapes
* Fixed a bug in Reshape check
* Added empty model path to Model constructor
* Made necessary changes to cmake to build from the binary package
* Changed INTEL_CVSDK_DIR to INTEL_OPENVINO_DIR
* Enable dyn device selection with C++ API
* Added Round operator to unsupported list
* Modified subgraph partition logic for MYRIAD
* Removed supported ops from the list
* Enable dyn dev selection in Py API's
* Add documentation for dynamic device selection
* Use MYRIAD || HDDL instead of VPU
* Removed temporary cast of Int64 to FP32
* Disabled unit Tests for CPU_FP32 and GPU_FP32
* Removed default "CPU" from unit tests to allow overriding
* Removed ops Concat, Squeeze, Unsqueeze from unsupported list
* Get the device id from info
* Removed overwriting device_id and precision
* Enabled ConvTranspose and EyeLike
* Reordered unsupported ops in alphabetical order
* Fixed syntax error
* Fixed syntax error
* Code clean-up: Handle exceptions, logs and formatting
Code formatted according to ORT coding guidelines.
* remove debug print from pybind code
* updated docs with ops and models
* formatting prints
* Added default values for c and j for openvino
* Overriding the values set for c and j to be 1
* BACKEND_OPENVINO should be empty if openvino is not in build
* Overriding c value with default for perftest
* fix VAD-M device string bug
* Add IE error details to exceptions
* Use IE specific device names in EP
* Add VAD-F (FPGA) device support
* Removed unecessary libraries from whl package
* Code changes for Windows compatibility
* Add VAD-F option to python API
* [revert before merge] cmake changes for RC
* Enable Windows build in CMake
* Unset macro OPTIONAL for windows builds
inference_engine.hpp's include chain defines a macro 'OPTIONAL'
which conflicts with onnx project's headers when using MSVC. So
would need to explictly unset it for MSVC.
* Use a single copy of plugin/IE::Core
Defined as a static member in Backend manager
* Remove restriction of single subgraphs for myriad
* Passed subgraph name to Backend to enhance log statements
* Disabled zero dimension conditions
* Disabled concat to remove zero dims
* Enabled building ngraph as part of ORT
* Removed serializing and added versioning
* Fix CPU_FP32 unit tests
* Removed unecessary condition
* add ngraph.so.0.0 to .whl
* Check for zero dimensions only for inputs and outputs
* Restrict loading only 10 subgraphs on myriad
* Build ngraph.dll within UEP. Doesn't link yet
* Rename Linux included libngraph.so to libovep_ngraph.so
Renames locally built libngraph.so containing ONNX importer to
libovep_ngraph.so in order to avoid linkage conflicts with
libngraph.so supplied by OpenVINO binary installer.
Applies only for Linux builds.
* use output_name cmake properties for lib name
* fix .so name format in lib_name.patch
* CMake code cleanup
* Rename WIN32 included ngraph.dll to ovep_ngraph.dll
To avoid conflict with ngraph.dll distributed by openvino.
* Added myriad config for networks without 4 dimensions
* Loading the 10 max clusters for inference on myriad
* Refactor code and add Batching support
Encapsulate subgraph settings into context structs.
Add batching support for completely supported models.
* Disabled some broken tests
* use input_indexes to avoid batch-checking initializers
* Avoid static initialization order error on WOS
* Added candy to broken tests
* InternalCI changes for 2020.2
* Updated DLDT instructions
* Unsaved changed in install_openvino.sh
* Changes after manual check
* Remove custom ngraph onnx_import build for WOS
ONNX Importer on WOS does not have protobuf issue.
* Remove FP32ToFP16 ngraph pass
This conversion is performed implicitly within IE.
* Surround debug logic by #ifndef NDEBUG
* remove invalid TODO comments
* removed references to ngrpah-ep
* clang-formatting
* remove commented code
* comment edits
* updating copyright year to that of first OpenVINO-EP release
* remove redundant log msg
* Modified operator and topology support
* Update build instructions
* doc formatting
* Fixed clip unit tests
* Revert "Remove FP32ToFP16 ngraph pass"
This reverts commit ec962ca5f315a5658ad980e740196f19de2639c1.
* Applying FP16 transformation only for GPU FP16
* Fixed GPU FP32 python tests
* automatically use full protobuf
* disable onnxrt server for now
* Disabled upsample
* update dockerfile instructions
* Removed MO paths and added ngraph path
* Remove OVEP from ORT Server docs
Will put it back in after validation
* Updated path to Ngraph lib
* Disabled Resize and some other python tests
* Removed unnecesary header files
* Use commit SHA to fetch ngraph repo
* Avoid un-needed file changes due to version update
* Fixed clip tests
* Fixed Pow, max and min onnx tests
* build.md doc typo
* Update cmake patch command for ngraph src
* remove dead cmake code for onnxruntime_USE_OPENVINO_BINARY
* use spaces instead of tab
* remove commented code
* Add info about protobuf version
* edit debug env var and enable for WIN32
* specify only version tag of 2020.2 for dockerbuilds
* remove unnecessary file changes
* Pass empty string as default argument to C# tests
* Use ${OPENVINO_VERSION} to name openvino install directory in CI builds
* Enabled unnecessarily disabled tests
* Fixed ngraph protobuf patch
* Fixed error in protobuf patch
* Revert "Use ${OPENVINO_VERSION} to name openvino install directory in CI builds"
This reverts commit 89e72adb8bf3b9712f5c81c5e13fe68c6c0df002.
* Remove unsetting OPTIONAL macro
This is no longer used in recent ONNX update onnx/onnx@da13be2,
so this unset workaround is no longer necessary.
* Use a null string default argument for C# API
* Set OpenVINO version yml files and pass to CI Docker builds
Git Tag info for DLDT as well as install directory are set
using this value.
This reverts commit 9fa9c20348ed72ae360a95c98e9b074d2f9fafc5.
* Documentation: recommendation and instructions for disabling ORT graph optimizations
* more doc updates
* Reduced the number of models according to CI time constraints
Co-authored-by: ynimmaga <yamini.nimmagadda@intel.com>
Co-authored-by: suryasidd <surya.siddharth.pemmaraju@intel.com>
Co-authored-by: Mikhail Treskin <mikhail.treskin@intel.com>
Co-authored-by: mbencer <mateusz.bencer@intel.com>
Co-authored-by: Aravind <aravindx.gunda@intel.com>
Co-authored-by: suryasidd <48925384+suryasidd@users.noreply.github.com>
* add frontend minst test
* to use torch nightly with torchvision
* remove incorrect comment per reviewer's comment
* experiment torchvision import failure
* experiment install_deps.sh
* more experiment install_deps.sh
* experiment install_deps.sh with --upgrade
* Experiment with install_deps.sh.
* Experiment with install_ubuntu.sh.
* Use Ubuntu 18.04 and Python 3.6 for CI.
* Update cmake version for CI.
* Install MPI on Ubuntu 18.04 for CI.
* Increase tolerance for MNIST test.
* Go back to Ubuntu 16.04 for CI, fix installing from deadsnakes ppa.
* Clean-up.
* Update ort_trainer.py from ort_training.
* Get default Ubuntu Python ver back to 3.5.
* Add underscore to opset_version parameter name in ORTTrainer constructor.
* Move loss/model wrap before the call for sample output.
* Update expected values for MNIST test.
Co-authored-by: liqun <liqun@OrtTrainingDev4.af05slrtruoetgaxwwjv5nsq5e.px.internal.cloudapp.net>
Co-authored-by: Sergii Dymchenko <sedymche@microsoft.com>
* Migrate winml to Microsoft Namespace (packaging changes are pending)
* add ns_prefix toggle
* fix packaging
* Users/sheilk/add missing raw header (#3484)
* add dualapipartition
* wrong variable for repo root
Co-authored-by: Sheil Kumar <sheilk@microsoft.com>
* remove existence check to force failures
* extra paren
* dualapipartition needs to be referenced from the source
* add microsoft.ai.machinelearning.dll to the output dir
* rename the idl file so that assembly info is correctly added into the winmd
* fix namespaces
* update namespaces
* default to microsoft, and add namespace override as build argument
* update cmakesetings.json as well
* remove from cmakelists.txt
Co-authored-by: Sheil Kumar <sheilk@microsoft.com>
Co-authored-by: Changming Sun <chasun@microsoft.com>
1. Fix static analysis warnings found by VC++
2. Add a new pipeline for static analysis
3. Merge all the windows CI build into one single yaml file.(Easier to queue them all).
4. Make DNNL build faster by disabling building the tests and examples.
5. Enable custom op unitest.
* Fix WCOS/Win32 linking bugs
* Remove unused NODEFAULTLIB flags
* Avoid plain target_link_libraries signature
* Avoid plain target_link_libraries signature
* Fix library list escaping
* Use library list instead of string
* Remove duplicate link to windowsapp.lib
* Remove Win32 build workarounds
* Specify CMake policies before initializing language
* Expose Win32 header definitions during build
* Force set API family
* Enable Win32 APIs in featurizer
* Use MT dynamic CRT
* Expose Win32 specific functions
* Disable app container globally
* Disable default wide functions in featurizers
* Add featurizers to test include path
* Workaround https://gitlab.kitware.com/cmake/cmake/issues/19428
* Revert pipeline debugging hacks
* Skip /FI in CUDA sources
* Default to Win32 builds
* Enable WCOS when using WinML
* Use generator expression to apply CMAKE_MSVC_RUNTIME_LIBRARY to C++ only
1. Fix onnxruntime server docker file build failure. Tested with the notebook in ONNX tutorial, it works well.
2. Delete the docker files for the other EPs, because currently they don't work and I don't have enough time to update them.
We want to implement SoftmaxCrossentropy and NegativeLossLikelihoodLoss forward training ops for opset-12 but that requires ONNX submodule to point to the latest commit to have the latest and greatest ONNX spec!
- Reverse integrate changes from *.in.proto files in github ONNX repo.
- Regenerate csharp/test/Microsoft.ML.OnnxRuntime.Tests/OnnxMl.cs
- Disable ONNX tests that don't have op implementation for the latest opset.
Discussed with Faith, because the data size is very small and changes are gradual, there is no need to delete the old data. We want to keep all the history.
Override native package name. Preserve managed package name the same.
Specify pckage name for validation purposes.
Fix up validation package name parameter.
Previously, we put the "bin" folder of all the CUDA verions in the system PATH. And 10.2 is in the front. It's a mess.
So I've removed all of them from the system PATH env. But I need to add one of them back through build scripts.
(The problem only affect the C# test, not the C/C++ tests that forked from build.py).
* add dml gpu pipelines
* add x86 to the gpu dml dev build pipeline
* Enable DML x86 builds
* Fix uint64_t -> size_t warning
* fix warnings
* enable dml on x86 ci builds
* operatorHelper 773 error uint32_t vs uint64_t
* operatorHelper 773 error uint32_t vs uint64_t
* make x86 pipeline use the gpu pool
* more warnings
* fix x86 directml path
* make dml nuget package
* disable tf_pnasnet_large
* disable zfnet512
* make validation use wildcards
* disable x86 dml gpu tests
* add args.
* update gpu.yml
* change nupkg wildcard
* add debug statements
* package x86 dml nupkg
* dont drop managed nuget again from dml pipeline build
* Add DML EULA
* directml license should be renamed to not clobber the existing license
* casing on dml package....
* {} to ()
* fix license name
* disable dml from x86 ci
* typo and cr feedback
* remove featurizers
* ship the dml pdb as well
* Initial commit
* More changes
* More changes
* More changes 3
* More changes 4
* More changes 5
* More changes 5
* More changes 6
* More changes 7
* More changes 8
* Remove C# ifdefs
* More changes 10
* More changes 11
* YAML changes for other release pipelines
* Add release notes metadata
* Props and Targets change
* Add CSHarp proj
* More changes 12
* More changes
* Minor fix
* Minor fix
* Fix yaml
* Some missing logic for winml
* Minor update
* Fix casing for winmd file
* Fix casing
* Add targets and props for managed section into native nuget
* revert file
* a
* Switch to CUDA10.2
* Update win-gpu-tensorrt-ci-pipeline.yml
* Update win-gpu-tensorrt-ci-pipeline.yml
* remove dynamic_shape
* update onnx-tensorrt submodule
* check if input shape is specified for TensorRT subgraph input and enable some TensorRT unit tests
* fix format issue
* add shape inference instruction for TensorRT
* update according to the reviews
* Update win-gpu-tensorrt-ci-pipeline.yml
* WIP: Re-enable x86 .NET testing in Release pipelines
Enabling x86 testing will make sure that ORT packages doesn’t break x86 projects of customers
* Remove setting some env variables
* Comment out a test failing on x86 builds
* More changes
* Minor fix
* More changes
* More changes
* s
* s
* s
* Revert minor change
* More changes
* More changes
* More changes 2
* explicitly set platform target
* Delete bin and obj folders
* Clean output dirs
* Add back TargetFramwork
* Disable x86 .net framework tests
* Skip x86 tests in MKLML pipeline
1. Add LTCG back. It was set to default OFF in my previous PR to speed up Windows build. It is only needed in release pipelines.
2. Remove --use_featurizers from all the packaging pipelines
3. Make sure all the packages have openmp
Use CUDA 10.1 for Linux build
(Windows change is already in)
Please note, cublas 10.2.1.243 is for CUDA SDK 10.1.243, not CUDA 10.2.x. CUDA 10.2.89 need cublas 10.2.2.89. They match on the last part of the digits.
libcublas10-10.1.0.105 won't work!!!
The cuda docker image by viswamy is already using 10.1, no need to change.
* update onnx-tensorrt submodule to trt7 branch
* add fp16 option for TRT7
* switch to master branch of onnx tensorrt
* update submodule
* update to TensorRT7.0.0.11
* update to onnx-tensorrt for TensorRT7.0
* switch to private branch due to issues in master branch
* remove trt_onnxify
* disable warnings c4804 for TensorRT parser
* disable warnings c4702 for TensorRT parser
* add back sanity check of shape tensort input in the parser
* disable some warnings for TensorRT7
* change fp16 threshold for TensorRT
* update onn-tensorrt parser
* fix cycle issue in faster-rcnn and add cycle detection in GetCapability
* Update TensorRT container to v20.01
* Update TensorRT image name
* Update linux-multi-gpu-tensorrt-ci-pipeline.yml
* Update linux-gpu-tensorrt-ci-pipeline.yml
* disable rnn tests for TensorRT
* disable rnn tests for TensorRT
* disabled some unit test for TensorRT
* update onnx-tensorrt submodule
* update build scripts for TensorRT
* formating the code
* Update TensorRT-ExecutionProvider.md
* Update BUILD.md
* Update tensorrt_execution_provider.h
* Update tensorrt_execution_provider.cc
* Update win-gpu-tensorrt-ci-pipeline.yml
* use GetEnvironmentVar function to get env virables and switch to Win-GPU-2019 agent pool for win CI build
* change tensorrt path
* change tensorrt path
* fix win ci build issue
* update code based on the reviews
* fix build issue
* roll back to cuda10.0
* add RemoveCycleTest for TensorRT
* fix windows ci build issues
* fix ci build issues
* fix file permission
* fix out of range issue for max_workspace_size_env
* Enable ARM64 release builds
* Add ARM release
* Skip C# dll signing in ARM
* Copy ARM binaries to Nuget
* Restore nuget packages before ARM packaging
* wip
* Use host protoc at C# build
* Set ProtocDirectory on cross-compiled builds
* wip
* Fix typo
* Initial Commit
* Merged PR 3985217: add onecoreuap_apiset.lib in order to avoid linking against kernel32.lib etc (#2346)
add onecoreuap_apiset.lib in order to avoid linking against kernel32.lib etc and violating our OS layering requirements.
We linked against onecoreuap_apiset.lib in VB so we will continue doing this, but I am still unsure why not to link against onecore instead since that is where we ship. However, since Sheil is the owner of this code we will wait to discuss with him before changing anything.
* Initial changes for layering
* more snipping to get core into ort
* update build instructions to include --build_shared_lib (#2358)
* update build instructions to include --build_shared_lib
* fix line breaks
* Task 23998197: add winml_lib_core into onnnxruntime.dll (#2368)
* Task 23998197: add winml_lib_core into onnnxruntime.dll
* PR feedback
build break on perf_test
* return proper error when the model path isn't found (#2391)
* LearningModelSession is cleaned up to use the adapter, and parts of b… (#2382)
this is a big PR. we are going to move it up to layer_dev , which is still a L3 so we are still safe to do work there agile.
we are going to move this into the L3 so that ryan can start doing intergration testing.
we will pause for a full code review and integration test result prior to going into the L2.
>>>> raw comments from previous commits >>>
* LearningModelSession is cleaned up to use the adapter, and parts of binding are.
* moved everything in the winmladapter
made it all nano-com using, WRL to construct objects in the ORT side.
base interfaces for everythign for winml to call
cleaned up a bunch of winml to use the base interfaces.
* more pieces
* GetData across the abi.
* renamed some namepsace
cleaned up OrtValue
cleaned up Tensor
cleaned up custom ops.
everything *but* learnignmodel should be clean
* make sure it's building. winml.dll is still a monolith.
* model moved over.
everything builds clean.
step !
* weak ref comment
* Layer dev paulm (#2408)
* model moved over.
everything builds clean.
step !
* weak ref comment
* added a wrapper for RoGetActivationFactory to hook back into winml for creating winml objects.
fixes model load.
* Layer dev paulm (#2414)
* model moved over.
everything builds clean.
step !
* weak ref comment
* added a wrapper for RoGetActivationFactory to hook back into winml for creating winml objects.
fixes model load.
* User/xianz/win ml telemetry (#2410)
* add option to enable winml telemetry
* add option to enable winml telemetry
* clean logs while developping
* clean the log of GUID
* compile onnxruntime_common with winml telemetry
* use option for use_telemetry
* rename option winml_use_telemetry to onnxruntime_use_telemetry
* little change
* fixed some lifetime management.
fixed the debug build.
squeezenet passes using winmlrunner for CPU and GPU
* Layer dev paulm (#2423)
* model moved over.
everything builds clean.
step !
* weak ref comment
* added a wrapper for RoGetActivationFactory to hook back into winml for creating winml objects.
fixes model load.
* fixed some lifetime management.
fixed the debug build.
squeezenet passes using winmlrunner for CPU and GPU
* PR feedback.
* Layer dev paulm (#2424)
* model moved over.
everything builds clean.
step !
* weak ref comment
* added a wrapper for RoGetActivationFactory to hook back into winml for creating winml objects.
fixes model load.
* fixed some lifetime management.
fixed the debug build.
squeezenet passes using winmlrunner for CPU and GPU
* PR feedback.
* couple of fixes and coded getmutabledata()
* Layer dev paulm (#2425)
* model moved over.
everything builds clean.
step !
* weak ref comment
* added a wrapper for RoGetActivationFactory to hook back into winml for creating winml objects.
fixes model load.
* fixed some lifetime management.
fixed the debug build.
squeezenet passes using winmlrunner for CPU and GPU
* PR feedback.
* couple of fixes and coded getmutabledata()
* fixed 2 more heap corruptions
* Layer dev paulm (#2426)
* model moved over.
everything builds clean.
step !
* weak ref comment
* added a wrapper for RoGetActivationFactory to hook back into winml for creating winml objects.
fixes model load.
* fixed some lifetime management.
fixed the debug build.
squeezenet passes using winmlrunner for CPU and GPU
* PR feedback.
* couple of fixes and coded getmutabledata()
* fixed 2 more heap corruptions
* Add opset and IR check when loading model (#2413)
* Add opset and IR check.
* Add test case for future opsets.
https://github.com/microsoft/onnxruntime/issues/2371
* fixed map and sequence when passing stl types across the ABI .
found a leak in nvidia driver, but skipped it.
all winmlapitests pass now
* Moved SessionOptions over to the abi
* WinML CI (#2412)
* Pass flags to build/test WinML in CI
* Add initial CMake config for unit tests in WinML
* Set winml_unittests standard to C++17
* Add WinML API tests and port them to googletest
* Install WinML test collateral
* Add LearningModelSessionAPITests ported to googletest
* Fix WinML test files encoding
* Add GPU tests
* Add parameterized test, skip GPU tests
* Enable precompiled header
* Remove unused code and collateral
* Remove brand images
* Add dllload.cpp
* Remove images not used in API tests
* Add LICENSE.md to image collaterals
* Add models with licenses
* Remove FNS Candy tests
* Add API test models
* Add ModelInSubdirectory
* Install collaterals post-build with copy_if_different, split common lib
* fix warnings
* Link to gtest_main
* Register WinML TraceLogging provider on Onnxruntime.dll (#2455)
* Register WinML TraceLogging provider on Onnxruntime.dll
* Add ifdef to make sure trace logging provider has telemetry option when LAYERING_DONE
* No need for ifdef for TraceLoggingOptionMicrosoftTelemetry
* PR feedback
* Move etw registration into lotus environment constructor and deresgister in lotus environment destructor
* Brianma/cpuwinml (#2466)
* allow building winml cpu without dml.
* Brianma/breaks (#2469)
* fix some more breaks
* learning model doesn't need lotusEnvironment and CPU shouldn't include dmlEP headers
* move dml checks out of winml and into the adapter
* better error handling
* Brianma/fi (#2470)
* learning model doesn't need lotusEnvironment and CPU shouldn't include dmlEP headers
* User/xianz/win ml telemetry (#2410)
* add option to enable winml telemetry
* add option to enable winml telemetry
* clean logs while developping
* clean the log of GUID
* compile onnxruntime_common with winml telemetry
* use option for use_telemetry
* rename option winml_use_telemetry to onnxruntime_use_telemetry
* little change
* Add opset and IR check when loading model (#2413)
* Add opset and IR check.
* Add test case for future opsets.
https://github.com/microsoft/onnxruntime/issues/2371
* WinML CI (#2412)
* Pass flags to build/test WinML in CI
* Add initial CMake config for unit tests in WinML
* Set winml_unittests standard to C++17
* Add WinML API tests and port them to googletest
* Install WinML test collateral
* Add LearningModelSessionAPITests ported to googletest
* Fix WinML test files encoding
* Add GPU tests
* Add parameterized test, skip GPU tests
* Enable precompiled header
* Remove unused code and collateral
* Remove brand images
* Add dllload.cpp
* Remove images not used in API tests
* Add LICENSE.md to image collaterals
* Add models with licenses
* Remove FNS Candy tests
* Add API test models
* Add ModelInSubdirectory
* Install collaterals post-build with copy_if_different, split common lib
* fix warnings
* Link to gtest_main
* fix bad merge
* Checking in a staging checkpoint point so that Ryan can work with me in parrallel
* build break.
* Brianma/testfails (#2473)
* add missing ir version to dictvectorizer-string.onnx
* add missing ir version to relu.onnx
* add missing ir version to zipmap*onnx
* add IR version to manually generated models
* remove an unnecessary ifdef dml
* Brianma/windowsai fi (#2475)
* update dockerfiles/README (#2336)
* Make elementwise op run 4 items per thread (#2335)
Description: Describe your changes.
Make elementwise op run 4 items per thread
unroll for loop to leverage ILP
remove unnessary N==0 check inside elementwise GPU kernel
Motivation and Context
Why is this change required? What problem does it solve?
It can improve the performance of GPU elementwise ops. ~2% performance gain on popular NLP bert model.
If it fixes an open issue, please link to the issue here.
* Add CUDA GatherElements kernel (#2310)
* Updates
* Update test
* Update
* Updates
* nits
* PR feedback
* Update
* Update
* PR feedback
* PR comments
* Update
* Fix build
* Fix build
* Nits
* Fix
* Layer Normalization Fusion (#2319)
basic layer normalization transform
* Add FastGelu Cuda Op for Gelu and Add bias fusion (#2293)
* Add FastGelu cuda op
* Add AddBiasGelu for experiment
* Revert "Add AddBiasGelu for experiment"
This reverts commit 5c1ee019858c657e6bb75887265cb85675626e5b.
* Add bias
* Add unit tests
* update comment
* update script
* fix build error
* update coding style
* update for CR feedback
Enable half2 optimization only when cuda arch >= 7.0
* move _Tanh to common.cuh
* implement CPU contrib OP Attention (#2333)
* Remove unused initializer from GraphProto as well as name_to_initial_tensor_ in CleanUnusedInitializers. (#2320)
* Remove unused initializer from GraphProto as well as name_to_initial_tensor_ in CleanupUnusedInitializers.
This means initializers that have been replaced during graph optimizations are not left in the GraphProto when we save an optimized model.
* Handle edge case where a model has an unused initializer with matching graph input by also removing the graph input.
* Use non-const iterators in std::find_if calls to make centos build happy.
* Nuget pipeline changes (#2305)
1. refactor the pipeline, remove some duplicated code
2. Move Windows_py_GPU_Wheels job to Win-GPU-CUDA10. We'll deprecated the "Win-GPU" pool
3. Delete cpu-nocontribops-esrp-pipeline.yml and cpu-nocontribops-pipeline.yml
4. In Linux nuget jobs, run "make install" before creating the package. So that extra RPAH info will be removed
* Cuda Reverse Sequence Op, maping types of same size using same template function. (#2281)
* Set ElementType to String type of node metadata, instead of byte[] (#2348)
* Set ElementType to String type of node metadata, instead of byte[]
* Fix spacing
* Introduce PrimitiveType into a Type System along with an integer constant (#2307)
Improve perf by avoiding GetType<T>() calls. Introduce MLTypeCallDispatcher to switch on Input Type. Add Tensor IsType<T>() fast method.
* Fix/test dim value of 0 handling in a couple of places (#2337)
* Update the CUDA Where implementation broadcasting logic to handle a dim with value of 0.
Add unit test
Also add unit test for unary op with dim value of 0
* Exclude ngraph from Where test with 0 dim.
* Openvino EP R3.1 onnxrt server (#2357)
* onnxrt server with OVEP
* onnxrt server with OVEP
* Update Dockerfile.server.openvino
* onnxrt server OVEP fix reviews
* onnxrt server OVEP fix reviews
* Implement cuda nonzero op. (#2056)
Implement cuda nonzero op.
* Direct use python numpy array's memory if already contiguous. (#2355)
* Direct use python numpy array's memory if already contiguous. This
could greatly improve performance for session with large input,
like big image 1920x1080 fastrcnn, 30~40% speed up could be achieved.
* Add test case enforce contiguous/non-contiguos numpy array as inputs.
* Add helper to create output to minimize binary size. (#2365)
Add ConstEigenTensorMap typedef so we don't unnecessarily const_cast the const input Tensor.
* fix builds enabling onnxruntime_DEBUG_NODE_INPUTS_OUTPUTS (#2369)
* fix builds enabling onnxruntime_DEBUG_NODE_INPUTS_OUTPUTS
* update
* Add Tracelogging for profiling (#1639)
Enabled only if onnxruntime_ENABLE_INSTRUMENT is ON
* test bidaf with nuphar for avx target (#2370)
increase nuphar test coverage a bit
* Fix a bug in TLS refcount that may destabilized CUDA CI (#2374)
* update output size calculation for resize (#2366)
* change how output size is calculated for resize op
* add tests for ver 10 resize
* Extend OneHot CPU kernel to support more types (#2311)
* Extend OneHot CPU kernel to support input int64_t, depth int32_t, output float
* Skip BERT before the test data fix is picked up
* Fix bug with Slice. Need to pass in flattened input dimensions so the initial offset into the input is calculated correctly. (#2372)
* Add opset 11 version of Split to CUDA ops (#2376)
Organize the CUDA ops definitions so all the opset 10 and 11 parts are together (same setup used for CPU ops)
* Layer Norm Fusion Fix (#2379)
* layer norm fusion fix
* Add input shape check in code and unit tests
* Fuse Add + Gelu (#2360)
Implement the transformer to fuse add + gelu
Implement the accurate kernel
* Skip layer norm transform (#2350)
* skip layer normalization transformer
* Another try to stabilize CUDA CI (#2383)
The root cause seems to be failure in CUDA dealloc when tear down. cudaFree return code was ignored before, so should the debug check.
* fix BUILD.md typo (#2375)
build.py: error: argument --config: invalid choice: 'RelWithDebugInfo' (choose from 'Debug', 'MinSizeRel', 'Release', 'RelWithDebInfo')
* Fixed compilation with ngraph (#2388)
* Fix reuse logic in allocation planner. (#2393)
* Fix reuse logic in allocation planner.
* PR comments
* Add helpful comments
* Don't allow reuse across string tensors.
* [NupharEP] Multiple optimizations (#2380)
Fuse transpose into MatMul
Implement Pow and constant scalar simplification
Vectorize ReduceMean
Improve symbolic shape inference
Minor updates for better debugging in fused function name
* Avoid using the default logger in the graph lib and optimizers (#2361)
1. Use the session logger if it is available.
2. Don't disable warning 4100 globally. We should fix the warnings instead of disabling it.
* Change CUDA implementation of Transpose to support all fixed size tensor types (#2387)
* Change CUDA implementation of Transpose to not use a typed kernel so we can support more types with minimum binary size.
Add support for 8, 16, 32 and 64 bit types.
Add unit tests.
Add method so the implementation can be called directly (will be used by CUDA Scan very soon).
* Disable TensorRT for MLFloat16 and int8 unit tests.
* Address PR comment and add support for calling cublas implementation if type is mlfloat16.
* Add opset 11 versions of the existing CUDA operators that had negative axis support explicitly added. (#2398)
* Add opset 11 versions of the existing CUDA operators that had negative axis support explicitly added.
* [NupharEP] force some low/zero cost ops to be inlined (#2409)
* fix cross compile bug (#2415)
* Minor optimization: if a node has already been placed, there's no need to find a kernel for it. (#2417)
* Add Reshape Fusion (#2395)
* Add reshape fusion
* Add some comments
* update comments
* update comment format
* update according to feedback
* update for recent logger change
* fix build error
* (1) Support both input and output edges in find path in graphutils
(2) Add a test case of only one constant initializer of Concat input.
(3) Refactor ReshapeFusion class to allow add more subgraph fusion in the future.
* fix error
* (1) loose constraint on initializer: non constant is allowed for reshape fusion.
(2) Change versions type to vector.
(3) Add logging.
(4) Return false when multiple output edges matched in FindPath. Add comments.
* only allow one direction (input or output) in FindPath
* [NupharEP] Update notebook and docker image (#2416)
Add BERT squad in Nuphar tutorial
Enhance speed comparsion readability
* Fix the issue in matmul_add_fusion (#2407)
Fix the issue in matmul_add_fusion
If Muatmul + Add has shape [K] * [K, N], reset it to [1, K] * [K, N] will make the output shape to [1, N] will also requires a reshape on the output.
Fix: just remove the shape reset to not fuse it.
Add a negative test case for matmul+add fusion
* feat(treeregressor): Update TreeEnsembleRegressor for type support (#2389)
Updates the `TreeEnsembleRegressor` to allow for `double`, `float`,
`int64`, and `int32` inputs to match the upstream specification.
Signed-off-by: Nick Groszewski <nicholas.groszewski@capitalone.com>
* onnxrt server documentation update (#2396)
* Added support for Pad-2 operator in OpenVINO-EP (#2405)
* Add CUDA If operator. (#2377)
* Add CUDA If operator.
Uses CPU operator for implementation.
By adding a CUDA version the inputs/outputs (with the exception of the 'cond' input) stay on GPU, and no other logic is required to avoid a copy to CPU across the control flow node.
* Improved documentation for onnxruntime::utils::SwapByteOrderCopy(), added precondition check.
* Fix the type constraints on CUDA If operator to exclude strings. (#2431)
* add Im2col<uint8_t> (#2438)
* Adjust codegen vectorization width from target (#2439)
* Adjust codegen vectorization width from target
* Add CUDA Scan operator. (#2403)
* Add Scan CUDA op.
Uses CPU implementation for logic.
Added some device specific functors for handling when data needs to be manipulated on a different device.
Added ability to override the materialization logic in the OrtValue slicer so DML can plugin their handling.
* Fix Windows GPU C API packaging pipeline failure (#2440)
Fix Windows GPU C API packaging pipeline failure (#2440)
* Correctly handle implicit inputs for fused nodes (#2390)
* Correctly handle implicit inputs for fused nodes
Previously, nuphar's partitioning function didn't include
node's implicit inputs into the inputs list of MetaDef, and hence
a crash was triggered in the onnx graph checker.
This commit fixed the issue. Furthermore, it also fixed a related
issue where we didn't add implicit inputs into
graph_inputs_excluding_initializers_ in Graph::SetGraphInputsOutputs.
the issue was that graph_inputs_including_initializers_ populated by
SetInputs (e.g. called by FunctionImpl::FunctionImpl) may contain
implicit inputs which were not of any node's initializers in the graph.
Because they were not part of any initializers, these implicit inputs
couldn't be visited by going through all nodes' inputs.
Consequently, they would *not* be added into graph_inputs_excluding_initializers_.
We fixed the issue by first copying the populated graph_inputs_including_initializers_
into graph_inputs_excluding_initalizers_, which then had both initializers and
non-initializers as its initial content. Later, we erase initializers from the
list. In this way, we can ensure all implicit inputs to remain in
graph_inputs_excluding_initializers_.
* refined comments and fixed duplicates
Address CR by revisiting comments in terms of implicit inputs
Also fixed an issue by skipping duplicates while copying inputs
from graph_inputs_including_initializers_.
* address CR
explain why we need to collect nodes' implicit inputs
* don't rely on pointer values for iterating std::set
Previously, openvino relied on iterating a set of NodeArg pointers
to construct inputs and outputs for a fused graph. It could cause
non-determinism. The reason was that although iterating std::set by
itself is stable, pointer values of NodeArgs may vary. Consequently,
we could end up visiting the set's elements in different orders for
different runs for the same test, which resulted in constructing
inputs (and outputs) with different orders to the fused graph.
For example, for the same test, we may have inputs [A, B] in some
runs but inputs[B, A] in others.
Let's use std::string as the key type to avoid such nondeterminism.
This commit also added implicit inputs into meta->inputs while returning
the capability from the openvino provider.
* Fixed another latent issue in openvino's GetCapability function
The issue was that we couldn't simply erase fused_inputs and fused_outputs
while iterating the nodes. For example, an output NodeArg may have multiple
uses, and it's wrong if we erase it from fused_outputs when we encounter only
one of its uses as input.
* Remove DeviceAllocatorRegistry class (#2451)
Remove DeviceAllocatorRegistry class
* CSharp api and test for loading custom op shared library (#2420)
- Added C-API test for loading custom op shared lib.
- Made some changes in C++ api header and C-api implementation to get it working.
- Added C# API and corresponding test for loading custom op shared library.
* Parallel Gelu with ParallelFor (#2399)
Parallel Gelu to get better performance for Gelu
* Clean up build.py (#2446)
* Pull the latest image before running docker build
* Fuse SkipLayerNorm with Bias (#2453)
Fuse SkipLayerNorm with Bias
* Allow more than one invocation of CreateEnv in the same process. (#2467)
* Allow more than one invocation of CreateEnv in the same process.
* Fix centos build
* Symbolic shape inference improvements: (#2460)
* Symbolic shape inference improvements:
- add a mode to guess unknown ops' output rank
- add support for GatherND
- add support for If
- fix a bug in get_int_values when then tensor rank > 1D, by treating it as no sympy data
- add symbol to literal merge when ONNX silently merges dims
- fix a bug in Concat when input dim is 0
- fix a bug in ConstantOfShape that computed dim is not updated
- add support for dynamic shape in ConstantOfShape
- fix a bug in Loop output shape that loop iterator dim is not inserted at dim 0
- add support for dynamic padding in Pad
- add support for dynamic shape in Reshape
- add support for Resize with opset > 10, by treating output dims as dynamic
- fix a bug in Slice when starts/ends are dynamic
- restrict input model to opset 7 and above
- make output model optional to avoid disk write when testing
Run model tests for symbolic shape inference
Reduce 2GB docker image size of nuphar
* add additional test data set for nuget pipeline (#2448)
* add SAS token to download internal test data for nuget pipeline
* update azure endpoint
* fix keyvault download step
* fix variable declaration for secret group
* fix indentation
* fix yaml syntax for variables
* fix setting secrets for script
* fix env synctax
* Fix macos pipeline
* attempt to add secrets to windows download data
* fix mac and win data download
* fix windows data download
* update test data set url and location
* Revert "Brianma/windowsai fi (#2475)"
This reverts commit 5780b864a1.
* Add scenario tests (#2457)
* Add scenario tests
* Remove TODO from model license
* Add winml_api test dependency
* fix model load test. fi from master changed the constructor (#2483)
* make api tests all pass (#2486)
* fix bad merge
* fix bad model merge
* Layer dev paulm (#2492)
* commetns for dml graph transformer
fixed ort value passing using the allocatir info
* fixed and coded maps and sequences across the abi
* Rename ambiguous header (#2489)
* fix one more missing IR version model (#2500)
* add missing IR version to 4 more models used by scenario tests (#2501)
* Add CLI parameters to test runner, build WinML in ARM and x86 CI (#2479)
* Support test parameters through CLI arguments
* Add WinML do Windows x86/ARM CI builds
* Code style fixes
* Update googletest
Remove GPUTEST macros everywhere now that GTEST_SKIP is supported
* Refactor main.cpp
* Build scenario tests without DML
* Link scenario tests to DML when it's enabled (#2502)
* Layer dev release pipeline (#2488)
Adds winml binaries to existing cpu nuget package, and creates new gpu dml nuget package with winml binaries and DML EP.
* Layer dev paulm (#2506)
* commetns for dml graph transformer
fixed ort value passing using the allocatir info
* fixed and coded maps and sequences across the abi
* cleaned up w4's
cleaned up the model info ABI
delayload directml.dll from winml
* Remove usage of IOBinding in WinML and use C_API Run method (#2504)
* remove usage of iobinding
* Change data structure to use vector of Ort::Values
* Polish bind input / output
* Use C APIrun method
* Update providers on evaluate getresults
* Remove run and IObinding interface from WinMLAdapter
* Remove use of IObinding
* bind unbound outputs code moved to learningmodelbinding
* clean up unneeded istensor adapter function
* Fix comment
* Check if session is closed before binding and clearing
* PR feedback
* Layer dev paulm (#2507)
* commetns for dml graph transformer
fixed ort value passing using the allocatir info
* fixed and coded maps and sequences across the abi
* cleaned up w4's
cleaned up the model info ABI
delayload directml.dll from winml
* cleaned up namepsace aliases.
renamed _winmla to winmla
this was good PR feedback from tiago a while back.
* Make tests dependend on winml_dll (#2509)
* add dml binaries to DirectML package and be more explicit about condition variables (#2520)
* re-enable warnings for winml builds and fix the warnings that were hiding (#2526)
* turn devmode back on for winml builds
* fix some warnings. include protobuf in a way that disables some warnings
* undo protobufhelpers changes and just ignore 4100 errors in pb code
* attempt to isolate protobufhelpers errors
* add template specialization for getting tensor proto data
* Layer dev paulm (#2533)
* commetns for dml graph transformer
fixed ort value passing using the allocatir info
* fixed and coded maps and sequences across the abi
* cleaned up w4's
cleaned up the model info ABI
delayload directml.dll from winml
* cleaned up namepsace aliases.
renamed _winmla to winmla
this was good PR feedback from tiago a while back.
* moved files from inc to lib\api.core
cleaned up some of the cmake
* staged changes
* Spawn child process to run DeviceLostRecovery scenario test (#2530)
* Spawn child process to run DeviceLostRecovery scenario test
* Layer dev paulm (#2536)
ori said yes
* add missing namespace to winml_trace_logging_provider in lotusenvironment.h (#2542)
* Handle exception thrown from all apis in WinMLAdapter (#2539)
* various changes to unblock windowsai ADO build
* Fix custom ops scenario tests (#2562)
* Do not shutdown protobuf after ort environment gets destroyed. Lazy load lotus environment first time it is needed
* comment typo
* pr comment about calling phoenix singleton
* Make lotus_environment static in winmladapter
* Layer dev paulm (#2567)
* commetns for dml graph transformer
fixed ort value passing using the allocatir info
* fixed and coded maps and sequences across the abi
* cleaned up w4's
cleaned up the model info ABI
delayload directml.dll from winml
* cleaned up namepsace aliases.
renamed _winmla to winmla
this was good PR feedback from tiago a while back.
* moved files from inc to lib\api.core
cleaned up some of the cmake
* staged changes
* making windowsAI azure dev ops work.
* code review comments.
* revert changes
* Cmake and preprocessor fixes that where uncovered by building on agents without DML available via SDK
* Layer dev dml delayload (#2580)
* Brianma/cpu (#2583)
* don't include dml stuff in cpu builds
* tests that link the image lib also need the telemetry lib now
* Throw Winml_err_invalid_binding if binding gpu resource on cpu device (#2589)
* Throw Winml_err_invalid_binding if binding gpu resource on cpu device
* PR comments. No need to query executionprovider for is gpu device
* User/xianz/ortthrow (#2596)
* thrown and handle onnxruntime exceptions
* handle exception thrown from ort in winmladapter
* undo changes in error.h
* add message to HRESULT
* User/xianz/ortthrow (#2599)
* thrown and handle onnxruntime exceptions
* handle exception thrown from ort in winmladapter
* undo changes in error.h
* add message to HRESULT
* add status error message
* Remove uwp onsuspending winrt call because logruntimeperf is getting removed (#2630)
* User/xianz/dedup telemetry (#2631)
* investigate duplication of telemetry in winml and ort
* remove winml telemetry events
* telemetry executionProviderEvent
* remove unneccessary file and refactor code little bit
* Revert back TelemetryEvent, which send up ETW event.
* merge changes from layer_dev to windowsai (#2638)
* Remove underscore from googletest names (#2616)
* Fix leaking memory allocator
Fix https://microsoft.visualstudio.com/OS/_workitems/edit/24278761
and https://microsoft.visualstudio.com/OS/_workitems/edit/24330198
* Explicitly initialize Ort::Value with nullptr
* Cache WinML adapter
* bad merge
* define private version of dxcore enum that is added in 19H1 SDK. (#2654)
* add comment for explaning private definition of dxcore d3d feature level ennum value. (#2672)
* do not package directml.pdb for redist packages. (#2676)
* Fix leaking operator registry (#2645)
Fix https://microsoft.visualstudio.com/OS/_workitems/edit/24354916
* User/orilevari/windowsai master merge (#2674)
merge resolutions included pulling in telemetry logic that was merged to master and not windowsai and dereferencing InferenceSession::sessionstate now that it is a unique pointer
* Delete Ort Allocator in LearningModelBinding (#2653)
* Delete OrtAllocator in LearningModelBinding
* PR comments to make Ort::Allocator a smart pointer
* Small comment change
* PR feedback to clean up code
* PR feedback on move semantics
* Clean up std::move
* Fix memory leaks (#2679)
Fix https://microsoft.visualstudio.com/OS/_workitems/edit/24356109,
https://microsoft.visualstudio.com/OS/_workitems/edit/24388361 and
https://microsoft.visualstudio.com/OS/_workitems/edit/24388596
* various changes to properly organize and skip GPU tests. For now for No DML builds we will not run GPU tests at all. In the future we should adapt the tests to expect the appropiate errors. (#2695)
* Windowsai without fi (#2701)
* Disable Attention fusion tests when DISABLE_CONTRIB_OPS is defined (#2529)
* Setup java ci (#2528)
* Add provision in ORT for session options to be parsed when available via model file (#2449)
* Initial commit
* Fix gitmodules
* Nits
* Nits
* Updates
* Update
* More changes
* Updates
* Update
* Some updates
* More changes
* Update
* Update
* Merge
* Update
* Updates
* More changes
* Update
* Fix nits
* Updates
* Fix warning
* Fix build
* Add comment
* PR feedback
* PR feedback
* Updates
* Updates
* Update
* More changes
* Fix build break
* Comment test for now
* Updates
* Updates
* PR feedback
* Updates
* Nits
* Add tests
* Fix build
* Fix build
* Fix build
* Fix build break
* Fix build
* Nits
* PR feedback
* More change
* Expose GetSessionOptions in pybind logic and add unit test for python
* Fix build
* PR feedback
* PR feedback
* Revert "Disable thread pool creation when enabled OpenMP (#2485)" (#2535)
This reverts commit 7c7d5a149c.
* Add dynamic shape support in TensorRT execution provider (#2450)
* remove onnx-tensorrt submodule
* add new onnx-tensorrt submodule (experiment) for trt6
* update engine build for trt6
* update compile and compute for tensorrt6.0
* Update tensorrt_execution_provider.cc
* Update tensorrt_execution_provider.cc
* Update tensorrt_execution_provider.cc
* Update tensorrt_execution_provider.cc
* switch to onnx-tensorrt master for TensorRT6'
* Update tensorrt_execution_provider.cc
* Handle dynamic batch size and add memcpy in TensorRT EP
* update test cases
* Update tensorrt_execution_provider.cc
* update onnx-tensorrt submodule
* Update Dockerfile.ubuntu_tensorrt
* Update Dockerfile.ubuntu_tensorrt
* Update run_dockerbuild.sh
* Update run_dockerbuild.sh
* Update install_ubuntu.sh
* Update concat_op_test.cc
* Update tensorrt_execution_provider.cc
* Upgrade TensorRT to version 6.0.1.5
* Update onnxruntime_providers.cmake
* Update CMakeLists.txt
* Update reduction_ops_test.cc
* Update install_ubuntu.sh
* Update Dockerfile.ubuntu_tensorrt
* Update Dockerfile.tensorrt
* Update BUILD.md
* Update run_dockerbuild.sh
* Update install_ubuntu.sh
* Update onnxruntime_providers.cmake
* Update install_ubuntu.sh
* Update install_ubuntu.sh
* Update gemm_test.cc
* Update gather_op_test.cc
* Update CMakeLists.txt
* Removed submodule
* update onnx-tensorrt submodule
* update header file
* Removed submodule
* add submodule onnx-tensorrt kevin's branch shape-test'
* add debugging code
* Update tensorrt_execution_provider.cc
* Update tensorrt_execution_provider.cc
* merge master
* Removed submodule
* update onnx-tensorrt submodule
* add more changes for dynamic shapes
* Update tensorrt_execution_provider.cc
* update for dynamic shape
* update dynamic shape processing
* fix logger issue
* remove submodule onnx-tensorrt
* add submodule onnx-tensorrt
* add env variable min_subgraph_size
* remove redundency
* update document
* use onnxruntime::make_unique
* fix multi-run issue
* remove some tests to save CI build time
* Add dynamic shape test
* Update TensorRT-ExecutionProvider.md
* Add example of running Faster R-CNN model on TensorRT EP
* Add more details on env variables
* update environment variables
* Update tensorrt_basic_test.cc
* Update model tests
* Update tensor_op_test.cc
* remove --use_full_protobuf
* Update build.py
* User/xianz/telemetry (#2458)
* enabme telemetry
* enable telemetry
* set enable telemetry as default
* for debugging
* remove log and set disable telemetry as default back
* delete private file while testing
* resolve comment: mainly add license header, rename macro and update docs
* rewording in privacy.md
* Fix integer overflow in cuda NonMaxSuppression implementation (#2540)
* add test case that should pass but fail
* fix nms
* extract int_max_output_boxes_per_class
* Introduce container type runtime checks and other improvements (#2522)
Rework TensorSeq in a manner consistent with Tensor and SparseTensor
in terms of type system setup.
Reduce templating. Introduce helpers to ensure the same
data type.
Make OrtValue __dtor not virtual.
Introduce ContainerChecker
* Fix C API tests for centos and mac (#2544)
* change c++14 to c++11
* add ld lib path for centos
* enable csharp tests on macos
* fix C API test on MacOS + fix manylinux dotnet install
* fix manylinux dotnet install
* fix lib link
* Add back executable bit to build.py
* Fix a bug handling negative begin pad values in Pad op (#2550)
* Fix bug in Pad op
* Update
* DNNL CMAKE update (#2548)
* Fix android build (#2558)
* Update win-x86-ci.yml (#2557)
Fix build pipeline break
* Re-enable Windows C# tests (#2564)
* disable onnx_test_runner -x invocations for dnnl (#2568)
* Allow sequence length to be symbolic (#2559)
* setup java ci mac (#2570)
* make layernorm fusion to support opset 11 (#2545)
* Fix a warning found in the latest VS release
* Add more check on SkipLayerNorm and BiasGelu fusion (#2574)
* Fix file not found error during docker build. (#2569)
* Add ConvTranspose1D (#2578)
* Ryanunderhill/packagename test (#2582)
* [Nuphar EP] fixes for some object detection models (#2581)
Update notebook tutorial with multi-threaded int8 GEMM from #2517
* EmbedLayerNormalization Fusion Improvement (#2553)
Embedding layer norm fusion improvements - add more checks
* Update version (#2584)
* Temporarily exclude vgg19 test from Python backend test
1. temporarily exclude vgg19 test which comsumes too much memory, run out of memory on Upsquared device. Single test pass for vgg19, need furture investigation (#2588)
2. Update docker file to decrease the docker image size
* Update docs for Android NNAPI EP (#2586)
* Fix lto bug for protobuf and ubuntu
* add path to build dir before test run (#2590)
* Add missig env variables for mac pipeline test (#2595)
* Fixed an issue in updating realized dims (#2597)
when we update realized dims for scan's output, the sliced axis also
needs to be inclusive, i.e. we should check with "dim >= insert_inclusive_axis",
because the offset in the symbols are based on Scan sugraph.
Otherwise, we would end up with shape mismatch later.
* Java API for onnxruntime (#2215)
* Add support for opset 11 in reshape fusion (#2592)
Support opset verion 11 in reshape fusion
* Rename automl python tools folder to featurizer_ops. (#2593)
* Support opset 11 subgraph of Squad model in Embed Layer Normalization (#2605)
Support opset 11 Squad model that is exported from PyTorch nightly. The embed layer uses Range op which is missed in the transformer.
* symbolic shape inference: fix warnings in GPT-2 model (#2608)
And revise nuphar perf test on BERT squad
* Dump subgraph ID and fused graph ID (#2607)
* Dump subgraph ID and fused graph ID
Dump subgraph ID and fused graph ID for better debugging
* Remove local static fused_count
added a field global_fused_count_ to NupharExecutionProvider class
* EmbedLayerNormalization Fusion For Dynamic Squad Model Opset 10 (#2613)
Support subgraph of SQuAD model exported from pytorch with dynamic input axes
* Allow providers to be set for InferenceSession at construction (#2606)
* Remove unnecessary parameter in some places in GatherElements implementation (#2612)
* Remove unnecessary parameter in some places
* Update
* Update
* Make sure fenced tensor could not reuse other tensor. (#2561)
Fix random error caused by this.
* Improve Embed Layer Norm Fusion for SQuAD with static input shape (#2621)
* fix float16 comparison in initializer (#2629)
* epsilon attribute for layernormalization fusion (#2639)
* removed unnecessary batch file and fix path (#2640)
* Add shape inference to ConvTransposeWithDynamicPads schema (#2632)
* Improve cuda expand() opeator's performance. (#2624)
* Cuda pad optimize when no padding is needed. (#2625)
* Shortcut cuda Pad() when no padding is needed.
* Optimize cuda scatter() on 2D compatible. (#2628)
* Optimize cuda scatter() on 2D compatible.
* Add some comments.
* fix build error for ARM (#2648)
* Improve performance of resize() in Nearest mode (#2626)
Special treatment for 2D, check same size as input image.
And in 2d kernel, template use_expolation.
* Fix memory exception in Layer Norm Fusion (#2644)
* Windows CI changes(#2650)
* Revert "User/orilevari/windowsai master merge (#2674)"
This reverts commit fe26146311.
* Revert "Windowsai without fi (#2701)"
This reverts commit 285d4c85ff.
* Revert "User/orilevari/windowsai master merge (#2674)"
This reverts commit fe26146311.
* Deref unique pointer for session_state
* send shutdown event when dll is unloaded and EvaluationStop, SessionC… (#2704)
* send shutdown event when dll is unloaded and EvaluationStop, SessionCreationStart Events.
* Add EvalutationStart Event
* add comment
* use correct type for for loop (#2755)
* ARM CI (#2759)
* Set ARM agent pool
* Set CMake generator to VS 2019 in ARM
* Use system-wide CMake instead of custom version
Our custom version is too old for VS 2019
* Use DML and build shared lib in ARM CI
* Restore nuget packages in ARM CI
* Disable DML
* Refactor ARM debug/release builds
* Use system packaged Python version
* Remove hardcoded Python path
* Downgrade Python to 3.7 for build
* Remove explicit CMake path
* Fix invalid JSON in cgmanifest.json (#2760)
* Fix cgmanifest.json generating script (#2770)
* Fix protobuf submodule name
* Workaround pygit2 bug
* Remove usage of WHOLEARCHIVE in WinML CMake and add WinMLAdapterFactory (#2726)
* Remove usage of WHOLEARCHIVE in WinMLAdapter CMake and add WinMLAdapterFactory
* PR feedback, no need for dll(export) since using def file
* PR comments
* Small comment in gen_def.py
* User/orilevari/32bit comparison warning (#2800)
* use correct type for for loop
* explicitly specify void for parameters of OrtGetApiBase because the function is defined in c, so when the function is just (), it is interpreted as having an unknown number of parameters. This was causing compiler warning C4276.
* Move winml_provider_factory.h to proper location (#2801)
* Scneario Test : Build Google Test and Taef Test based on preprocessor definition (#2809)
* Add winml macro wrappers on top of google test macros
* change test methods to disabled
* Add custom winml macros for both taef and google tests
* PR comments
* Filter CPU case for IsFloat16Supported (#2802)
* Merge fixes
* CMake cross-generator fixes (#2790)
* Fix compilation w/ non-VS CMake generators
* Fix custom WINMD target in Ninja
* Remove usage of msbuild .targets file
* Fix linking using DML in Ninja
* Automate SDK kit version choice
* Cleanup DML package install
* Fix SDK version detection
* Fix comment
* Revert unittest linkage changes
* Fix latest SDK detection
* Don't link to non-uapcore libraries
* Remove MessageBoxA reference and unused link libs
* Refactor WinMLAPI Tests to build both google and taef test based on preprocessor definition (#2829)
* Add winml macro wrappers on top of google test macros
* change test methods to disabled
* Add custom winml macros for both taef and google tests
* PR comments
* Refactor winml api tests
* Move additional gtest specific macro definition into googleTestMacros.h
* Fix test build break since winml_lib_api needs to be statically linked to tests since winmlp::learningmodeldevice::iscpu() is being used in devicehelpers.cpp (#2837)
* Enforce WINML_TEST_CLASS_BEGIN_* matches w/ a WINML_TEST_CLASS_END (#2841)
* Fix warnings that cause build to fail
* Fix test warnings and delayload linking (#2843)
* Ortmemoryinfo struct changed
* mark the camera scenario test as edgecore because it uses d3d11 (#2852)
* User/orilevari/pipeline fi breaks (#2853)
* remove conflicting artifact names. Decided to stop using drop-nuget-cuda since this may have implications on other dependent pipelines.
* change job name in gpu.yml back to Windows_CI_GPU_CUDA_Dev
* Remove internal libs from tests (#2864)
* Support custom DML in onnxruntime_providers.cmake (#2867)
* Make DML include path global (#2882)
* Make DML include path global
* Add generated cppwinrt headers to winml_lib_common
* Integrate changes to WindowsAI to make ADO Build (#2886)
* Revert "CMake cross-generator fixes (#2790)"
This reverts commit dbe7d97fa1.
* add additional suppress warning in onnx_proto
* ignore /wd4996 warning
* DML execution provider fixes
* Revert "Revert "CMake cross-generator fixes (#2790)""
This reverts commit 1ae7b4bcbc.
* Update func signature of custom op function overloads
* common devicehelpers fixes
* Add pch.h for winml_lib_common
* re-add winml_lib_common_dir/inc to include path for winml_adapter
* User/orilevari/dml redist shared folder (#2890)
* move dml nuget package directory up one level to make it shared between build flavors
* Merge conflict fix
* Revert "Merge conflict fix"
This reverts commit 142fa72cf9ce4344ad717b50b7ea2b8582aadc7c.
* Revert "Merge remote-tracking branch 'origin/master' into windowsai"
This reverts commit 6e2126d46e5e5f564d65da37dd4f70c93dd81165, reversing
changes made to b3f5583dc9249834b947c8ea905f6a98060d5bd6.
* Make winml_test_common free of test macros (#2902)
* Add option to build winml_test_common without googletest specifics
* remove test macros from squeezenet
* comment change
* Make cmake functions to get scenario and api source
* PRcomments about hresult
* Build errors fixed
* Fix cmake variable
* Make winml_google_test_lib to build main.cpp once
* PRcomments
* Don't generate files outside the build root (#2914)
* Don't generate files outside the build root
* Add onnxruntime_EXTERNAL_DEPENDENCIES to WinML
* Add DML depedency on RESTORE_PACKAGES
* User/orilevari/fix yaml merge bugs (#2918)
* Add winml test source parameter into cmake function (#2919)
* Add option to build winml_test_common without googletest specifics
* remove test macros from squeezenet
* comment change
* Make cmake functions to get scenario and api source
* PRcomments about hresult
* Build errors fixed
* Fix cmake variable
* Make winml_google_test_lib to build main.cpp once
* PRcomments
* Add arguments to unittest cmake functions
* remove comment
* Revert "Revert "Merge remote-tracking branch 'origin/master' into windowsai""
This reverts commit ade5abe72a4234fdbc3623093c61c02c6b0bdc26.
* Fix breaks from merge with ORT master
* Brianma/linux (#2917)
* don't include windows.h in cross-plat header
* add default case for switch statement
* signed/unsigned mismatch fix
Co-authored-by: Brian Martin <42186431+martinb35@users.noreply.github.com>
* User/sheilk/winml adapter c api (#2891)
* Create winml adapter c api
* fix build
* make it build
* move adapter into onnxruntime core/session
* entry point not exported
* minor changes
* make model metadata work
* make tests pass
* implement all the model reflection apis on the adapter c abi
* update the new ort interface to create a lotus ennvironment with a logging sink
* start adding ort env
* move all winml code into adapter folder/lib to isolate it
* ensure a single logging manager at a time
* start refactoring session
* refactor session creation interface
* add cpu and dml session option methods to adapter
* finish session init
* stub out interfaces in ort lib to perform similar mechanics of iinference session
* enable profiling, and enable schema override
* update session register graph transformers
* turn back on custom registry for custom ops
* Add sync api
* add last c api stubs
* should build... but all feature values are broken since this is in flight to moving all implementation details into ivalue
* remove ep adapter header
* Implement DML execution provider functions from adapter (#2846)
* Implement DML execution provider functions from adapter
* Use functions in OnnxruntimeEngine.cpp
* make map/sequence type_infos freeable, and start implementing ivalue
* make it build again
* implement value methods
* implement remaining methods
* remove com adapter abi
* check dml session
* cache the allocator on ivalue
* check if resource is cpu/gpu when access its mutable data
* update tensor
* mismatched parentheses
* fix tensor base and binding obj
* it evaluates tensors! sometimes...
* minor fixes
* enable gpu evals
* wrapper all existing winml adapter apis with API_IMPL to try catch (#2854)
* update winml... tensor strings are broken, need to template tensorbase to do different things for strings
* make tensor strings work with 2 copies in/2 copies out
* Fix tensor string and allocator bug
* make maps work again... needs some fixes still
* Make it build!
* enable map inputs
* map outputs
* unbound outputs for sequences and maps
* User/xianz/merge windowsai (#2883)
* Packaging pipeline changes for VS 2019 (#2711)
* Tiny fix to codegen
* Simplify cache implementation and avoid static variables that may carry over between models
* Extend DML kernels (#2641)
* Additional DML operators
* Check unsupported attributes and inputs
* Address PR comments
* Add kernel capability function used for partitioning, and re-enable stride-based int64 support based on value range
* Fix test failures
* Build fix
* PR comments
* Update Nuphar tutorial notebook (#2721)
1. Reflect int8 GEMV improvements for multi-threading from #2696
2. Add notes on multi-threading control using OpenMP
3. Add samples of running multi-isa AOT, and show int8 GEMM differences between AVX and AVX2
4. Add rnn_benchmark example to resolve#1993
* Add schema for new Qops (#2611)
* Add schema for new Qops
* adding shape inference + qlinearaveragepool
* plus review comments
* plus review comments
* updates per review comments
* plus review comments
* [server] Add supposed for model_name and model_version as cli parameter (#2708)
* remove 64bit warning message from python validation. (#2727)
* MLAS: ARM64 build fix (#2734)
fix bad usage of vreinterpret to cast vector element types
* Fix broken python docs links (#2740)
* Fix build on Mac OS (#2731)
mac os ld doesn't support --while-archive, correct option is -all_load
* fix ngraph wheel (#2737)
* fix ngraph wheel
1.1.0 onnxruntime_ngraph wheel doesn't work
* remove libdnnl.so in nGraph Libs
* make it easy to compare
* Split onnxruntime server to a separated folder (#2744)
* Fix build for Python 3.8 (#2747)
* Fix build for Python 3.8
* Update protobuf to 3.11.2 (#1928)
Update protobuf to 3.11.2 (#1928)
* Change default optimization level to All (from Basic) (#2745)
* change default optimization level to All (from Basic)
* fix test
* fix c# test
* Update numpy to 1.18 (#2758)
* Update numpy to 1.18
* Pipeline changes for python 3.8 (#2753)
1. Pipeline changes for python 3.8
2. Fix a regression in setup.py which was just introduced in the previous commit.
Please notice, we still haven't made python 3.8 + Windows + CUDA work.
* Add basic stacktrace output for posix debug builds. (#2749)
* [NupharEP] fix a race condition when multiple sessions running different models concurrently (#2772)
* Revert "Change default optimization level to All (from Basic) (#2745)"
This reverts commit 56bb503c2f.
* Fix typo in error message (#2736)
* Rename MKL-DNN to DNNL to fix broken link (#2730)
* Fix nightly build version number issue
* Pass BUILD_BUILDNUMBER to linux docker
* Disable featurizers in python packages
* Import more featurizers (#2781)
Make kernels non-template. Add input constraint for learnt data.
Add min_max_scalar_transformer, robust_scalar_transformer,
inputation_marker_transfomer, label_encoder_transformer,
missing_dummies_transformer along with tests.
Advance Featurizers library commit.
* Implement a more stable softmax (#2715)
* Implement a more stable SoftMax
e^x is represented as infinity if x is large enough, like 100.f. Infinity divided by Infinity is a NAN. Thus, softmax gets a NAN if one or more item are large enough.
A math transform as below is leveraged to get a stable softmax:
e^xi/(e^x1 + ...e^xn) = e^(xi - max) / (e^(x1 - max) + ... + e^(xn - max))
And for convenience, force max to 0.f if all xi are negative
* Contributing: Fix a typo (#2784)
* ACL EP GEMM improvements (#2780)
When it is posible we use a fully connected layer instead of the gemm implementation.
This will let the library use the best implementation based on the input data.
* ACL EP convolution improvements (#2774)
Added the optimized implementation for depthwise convolution for both ACL v19.02 and ACL 19.05.
Also the pointwise convolution seems to be more optimal in the CPU implementation so we opted for that instead.
* Add script for release Nuget validation (#2719)
* Initial commit
* Nits
* Disable a test temporarily
* Change working directory
* Test
* Add download python step
* Test update
* More changes
* Fix space issue
* Fix
* Verify nuget signing
* Fix
* Spaces
* PR feedback
* Nit
* Fix
* Fix
* Remove temporary changes
* add uint8 support to where op (#2792)
* Improve bert optimization script: (#2712)
(1) Move input int64=>int32 conversion to embed layer fusion.
(2) Output epsilon attribute for LayerNormalization fusion.
* add session creation time cost. (#2798)
* ML.NET team needs featurizers within a package (#2789)
Add auto ml featurizers to Windows, MacOS as well as to GPU packaging-pipelines.
* Initialize max of softmax with lowest of float (#2786)
* MLAS: update SGEMM threading parameters (#2808)
* add interface to copy batch tensors. (#2807)
* add interface to copy batch tensors.
* onnxruntime
* speed up Windows TRT CI (#2811)
* don't run cuda tests if building with tensorrt
* remove unnecessary build options for win trt ci
* refactor win gpu tensorrt ci yml
* --numpy_version=1.17
* update
* update
* azcopy and cuda path
* Update test data (#2356)
* Add timeseries imputer transformer featurizer kernel (#2813)
Make kernels non-template. Add input constraint for learnt data.
Fixup tests.
Add two more featurizers along with tests. Tests fail.
min_max_scalar_transformer
robust_scalar_transformer
Fix tests serialized stream by prepending version bytes.
Add inputation_marker_transfomer and the test.
Fix up float/double type designations.
Added label_encoder_transformer along with a test.
string_throw case is broken at the momement.
Fix labelencodertransfomer_test.cc string_throw case
Rename maxabsscalertransformer_test.cc
Add MissingDummiesTransformer along with the test.
Update manifest.
Add TimeSeriesImputerTransformer definition, implementation and tests
* Fix memory leak in TRT (#2815)
* fix memory leak issue
* revert EP_FAIL on enueueV2
* Add manifest missing comma
* Run static code analyzer on most of our code (#2817)
* Scneario Test : Build Google Test and Taef Test based on preprocessor definition (#2809)
* Add winml macro wrappers on top of google test macros
* change test methods to disabled
* Add custom winml macros for both taef and google tests
* PR comments
* update quantization doc (#2783)
* update documentation for quantization script
* plus some spell corrections
* Filter CPU case for IsFloat16Supported (#2802)
* update default optimization level + fix gemm_activation fusion (#2791)
* update defualt optimization level + fix gemm_activation fusion
* fix typo
* add unit test and incorporate review comments
* fix test comment
* Fix dnnl wheel package name (#2823)
* Append '-dnnl' to whl package name when --use_dnnl
* Update build.py
* Update Ubuntu & TensorRT version in README (#2820)
Dockerfile.tensorrt is using nvcr.io/nvidia/tensorrt:19.09-py3 as base Image, update Ubuntu and TensorRT version according to
https://docs.nvidia.com/deeplearning/sdk/tensorrt-container-release-notes/rel_19-09.html#rel_19-09
* Merge fixes
* Add OneHotEncoder and HashOneHotEncoder kernels. (#2830)
Add defs and imlementation for OneHotEncoders, adjuist date_time_transformer kernel and test.
Add OneHotEncoder kernel test.
Add HashOneHotVectorizerTransformer unit test.
This does not link due to multiple definitions of functions
that are included into header from a CPP file.
* Upgrade gtest to the latest version (#2827)
WinML would like to update the googletest submodule. They want some newer features (namely GTEST_SKIP to skip tests programmatically and be able to skip entire fixtures easily) and would need to update the submodule version.
However, because the new version of code hit a bug in gcc, even though the bug is already fixed in the latest gcc but we're using gcc 4.8.x and it won't get patched for the bug, so we have to do a compromise, change our code a little bit to make it work.
The gcc bug: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=51213
* Add support for int64_t for topk CPU. Fixes github issue #2806. (#2833)
* Ignore allocator type in ExecutionProviders allocator map. Make default initialization of OrtMemoryInfo more clearly invalid. (#2768)
* Remove allocator type from the key comparison in ExecutionProviders.
Remove usage of DummyArena as it's no longer necessary.
* Fix x86 tests where arena allocator is disabled.
Make initialization of OrtMemoryInfo clearer by adding Invalid enum value.
* Make OrtValueNameIdxMap::MaxIdx more intuitive.
* Convert ExternalProject Featurizers into git submodule (#2834)
Add git submodule for Featurizer library.
Update cmake to build for git submodule.
* add domain check for nodes + update documentation (#2831)
* Fix cgmanifest.json generating script (#2770)
* Fix protobuf submodule name
* Workaround pygit2 bug
* User/orilevari/32bit comparison warning (#2800)
* use correct type for for loop
* explicitly specify void for parameters of OrtGetApiBase because the function is defined in c, so when the function is just (), it is interpreted as having an unknown number of parameters. This was causing compiler warning C4276.
* CMake cross-generator fixes (#2790)
* Fix compilation w/ non-VS CMake generators
* Fix custom WINMD target in Ninja
* Remove usage of msbuild .targets file
* Fix linking using DML in Ninja
* Automate SDK kit version choice
* Cleanup DML package install
* Fix SDK version detection
* Fix comment
* Revert unittest linkage changes
* Fix latest SDK detection
* Don't link to non-uapcore libraries
* Remove MessageBoxA reference and unused link libs
* Fix Linux CUDA nuget packaging pipeline break
* Refactor WinMLAPI Tests to build both google and taef test based on preprocessor definition (#2829)
* Add winml macro wrappers on top of google test macros
* change test methods to disabled
* Add custom winml macros for both taef and google tests
* PR comments
* Refactor winml api tests
* Move additional gtest specific macro definition into googleTestMacros.h
* Fix test build break since winml_lib_api needs to be statically linked to tests since winmlp::learningmodeldevice::iscpu() is being used in devicehelpers.cpp (#2837)
* Enforce WINML_TEST_CLASS_BEGIN_* matches w/ a WINML_TEST_CLASS_END (#2841)
* update optimization doc for BERT related fusions (#2819)
* Add bert related transformers to doc
* Add execution provider and comment for bert optimizations
* Add comment about accuracy impact of approximation
* Fix warnings that cause build to fail
* MLAS: enable threading for quantized GEMMs (#2844)
* Fix test warnings and delayload linking (#2843)
* Ortmemoryinfo struct changed
* mark the camera scenario test as edgecore because it uses d3d11 (#2852)
* User/orilevari/pipeline fi breaks (#2853)
* remove conflicting artifact names. Decided to stop using drop-nuget-cuda since this may have implications on other dependent pipelines.
* change job name in gpu.yml back to Windows_CI_GPU_CUDA_Dev
* Remove internal libs from tests (#2864)
* Support custom DML in onnxruntime_providers.cmake (#2867)
* remove old winmladapter cpp
Co-authored-by: Changming Sun <chasun@microsoft.com>
Co-authored-by: KeDengMS <kedeng@microsoft.com>
Co-authored-by: Jeff <38966965+jeffbloo@users.noreply.github.com>
Co-authored-by: Ashwini Khade <askhade@microsoft.com>
Co-authored-by: Andrey <andrey.lompart@gmail.com>
Co-authored-by: George Wu <jywu@microsoft.com>
Co-authored-by: Tracy Sharpe <42477615+tracysh@users.noreply.github.com>
Co-authored-by: Faith Xu <txsafx@gmail.com>
Co-authored-by: zhanyi-ms <zhanyi@microsoft.com>
Co-authored-by: Changyoung Koh <gkcy1019@gmail.com>
Co-authored-by: Scott McKay <Scott.McKay@microsoft.com>
Co-authored-by: Takeshi Watanabe <take-cheeze@users.noreply.github.com>
Co-authored-by: Dmitri Smirnov <yuslepukhin@users.noreply.github.com>
Co-authored-by: Yufeng Li <liyufeng1987@gmail.com>
Co-authored-by: Maher Jendoubi <maher.jendoubi@gmail.com>
Co-authored-by: Andrews548 <32704142+Andrews548@users.noreply.github.com>
Co-authored-by: Hariharan Seshadri <shariharan91@gmail.com>
Co-authored-by: Nathan <7902510+ybrnathan@users.noreply.github.com>
Co-authored-by: Tianlei Wu <tlwu@microsoft.com>
Co-authored-by: Ke Zhang <kezhan@microsoft.com>
Co-authored-by: stevenlix <38092805+stevenlix@users.noreply.github.com>
Co-authored-by: Ryan Lai <ryalai96@gmail.com>
Co-authored-by: Ori Levari <ori.levari@microsoft.com>
Co-authored-by: Yingge WAN <y-wan@users.noreply.github.com>
Co-authored-by: Qing <cwq1913@gmail.com>
Co-authored-by: Pranav Sharma <emailpranav@gmail.com>
Co-authored-by: Tiago Koji Castro Shibata <tiago.shibata@gmail.com>
* move sequence implementation into ort lib... still commented out... need to turn back on...
* begin sequence implementation
* make maps and sequences work
* fix broken tests
* remove dead code
* misc cleanup
* CR feedback
* User/xianz/winml adapter c api (#2869)
* wrapper all existing winml adapter apis with API_IMPL to try catch
* Return HR or Throw for WinML adapter APIs if failed
* undo macro wrapper for two places
* Wrap error macros around ort apis, too.
* address CR feedback #2
* add more api throw/return macros
* Revert changes no longer needed
* revert changes to cxx api
* format winml lib.ort and winml adapter
* remove static pheonix singleton
Co-authored-by: Ryan Lai <ryalai96@gmail.com>
Co-authored-by: Xiang Zhang <xianz@microsoft.com>
Co-authored-by: Changming Sun <chasun@microsoft.com>
Co-authored-by: KeDengMS <kedeng@microsoft.com>
Co-authored-by: Jeff <38966965+jeffbloo@users.noreply.github.com>
Co-authored-by: Ashwini Khade <askhade@microsoft.com>
Co-authored-by: Andrey <andrey.lompart@gmail.com>
Co-authored-by: George Wu <jywu@microsoft.com>
Co-authored-by: Tracy Sharpe <42477615+tracysh@users.noreply.github.com>
Co-authored-by: Faith Xu <txsafx@gmail.com>
Co-authored-by: zhanyi-ms <zhanyi@microsoft.com>
Co-authored-by: Changyoung Koh <gkcy1019@gmail.com>
Co-authored-by: Scott McKay <Scott.McKay@microsoft.com>
Co-authored-by: Takeshi Watanabe <take-cheeze@users.noreply.github.com>
Co-authored-by: Dmitri Smirnov <yuslepukhin@users.noreply.github.com>
Co-authored-by: Yufeng Li <liyufeng1987@gmail.com>
Co-authored-by: Maher Jendoubi <maher.jendoubi@gmail.com>
Co-authored-by: Andrews548 <32704142+Andrews548@users.noreply.github.com>
Co-authored-by: Hariharan Seshadri <shariharan91@gmail.com>
Co-authored-by: Nathan <7902510+ybrnathan@users.noreply.github.com>
Co-authored-by: Tianlei Wu <tlwu@microsoft.com>
Co-authored-by: Ke Zhang <kezhan@microsoft.com>
Co-authored-by: stevenlix <38092805+stevenlix@users.noreply.github.com>
Co-authored-by: Ori Levari <ori.levari@microsoft.com>
Co-authored-by: Yingge WAN <y-wan@users.noreply.github.com>
Co-authored-by: Qing <cwq1913@gmail.com>
Co-authored-by: Pranav Sharma <emailpranav@gmail.com>
Co-authored-by: Tiago Koji Castro Shibata <tiago.shibata@gmail.com>
* missing use_dml check in winml_adapter_session (#2930)
* --use_dnnl flag was mangled in merge (#2931)
* use dml macro not wrapping custom registry code (#2934)
* Disable LNK4199 winml_dll to enable cuda builds (#2936)
* Disable LNK4199 in winml_dll
* linkler->linker
* LearningModelSessionAPITestGpu.CreateSessionWithCastToFloat16InModel should return DXGI_ERROR_UNSUPPORTED when FP16 not supported (#2937)
* Disable LNK4199 in winml_dll
* linkler->linker
* Need to return DXGI_ERROR_UNSUPPORTED when Model does not support fp16
* Publish build symbols (#2939)
* Publish build symbols
* Don't upload PDBs for .exe files
* Make x86 build (#2943)
* fix last remaining size_t/int64_t warnings->errors (#2948)
* TensorString, Sequences and Maps use the first allocator, but should use the cpu default allocator. (#2952)
* fix tensor string allcoator
* clean up default allocator usage for strings in winml lib/api.ort
Co-authored-by: Ryan Lai <ryalai96@gmail.com>
* Handle tensor shape of zero (#2954)
Co-authored-by: Ryan Lai <ryalai96@gmail.com>
* CR feedback (#2970)
* CR feedback
* fix weird formatting on privacy readme
* Add 'All rights reserved.' everywhere
* readd all rights reserved to winml_provider_factory.h
* remove extra space in comment
* remove extra whitespace
* fixes post master merge
* remove winml from nuget gpu pipeline
* set IR VERSION on generated_model in rnn_benchmark (#2972)
* Fix slice conformance failures (#2908)
Co-authored-by: Adrian Tsai <adtsai@microsoft.com>
Co-authored-by: Brian Martin <42186431+martinb35@users.noreply.github.com>
Co-authored-by: Ryan Lai <ryalai96@gmail.com>
Co-authored-by: Paul McDaniel <paul_mcdaniel@hotmail.com>
Co-authored-by: Xiang Zhang <xianz@microsoft.com>
Co-authored-by: Dwayne Robinson <fdwr@hotmail.com>
Co-authored-by: Tiago Koji Castro Shibata <tiago.shibata@gmail.com>
Co-authored-by: Ori Levari <ori.levari@microsoft.com>
Co-authored-by: Jeff <38966965+jeffbloo@users.noreply.github.com>
Co-authored-by: Changming Sun <chasun@microsoft.com>
Co-authored-by: KeDengMS <kedeng@microsoft.com>
Co-authored-by: Ashwini Khade <askhade@microsoft.com>
Co-authored-by: Andrey <andrey.lompart@gmail.com>
Co-authored-by: George Wu <jywu@microsoft.com>
Co-authored-by: Tracy Sharpe <42477615+tracysh@users.noreply.github.com>
Co-authored-by: Faith Xu <txsafx@gmail.com>
Co-authored-by: zhanyi-ms <zhanyi@microsoft.com>
Co-authored-by: Changyoung Koh <gkcy1019@gmail.com>
Co-authored-by: Scott McKay <Scott.McKay@microsoft.com>
Co-authored-by: Takeshi Watanabe <take-cheeze@users.noreply.github.com>
Co-authored-by: Dmitri Smirnov <yuslepukhin@users.noreply.github.com>
Co-authored-by: Yufeng Li <liyufeng1987@gmail.com>
Co-authored-by: Maher Jendoubi <maher.jendoubi@gmail.com>
Co-authored-by: Andrews548 <32704142+Andrews548@users.noreply.github.com>
Co-authored-by: Hariharan Seshadri <shariharan91@gmail.com>
Co-authored-by: Nathan <7902510+ybrnathan@users.noreply.github.com>
Co-authored-by: Tianlei Wu <tlwu@microsoft.com>
Co-authored-by: Ke Zhang <kezhan@microsoft.com>
Co-authored-by: stevenlix <38092805+stevenlix@users.noreply.github.com>
Co-authored-by: Yingge WAN <y-wan@users.noreply.github.com>
Co-authored-by: Qing <cwq1913@gmail.com>
Co-authored-by: Pranav Sharma <emailpranav@gmail.com>
1. Add support for vstest.
2. Add support for vcpkg. To use it:
```bat
vcpkg install zlib:x64-windows benchmark:x64-windows gtest:x64-windows protobuf:x64-windows pybind11:x64-windows re2:x64-windows
mkdir build
cmake ..\cmake -DCMAKE_BUILD_TYPE=Debug -A x64 -T host=x64 -DCMAKE_TOOLCHAIN_FILE=C:\vcpkg\scripts\buildsystems\vcpkg.cmake -DVCPKG_TARGET_TRIPLET=x64-windows -Donnxruntime_PREFER_SYSTEM_LIB=ON
```
3. New cmake option: onnxruntime_PREFER_SYSTEM_LIB, which allows user using the preinstall libs instead of the things in onnxruntime submodule.
4. New cmake option: onnxruntime_ENABLE_MEMLEAK_CHECKER, which allows user turn on/off the memory leak checker by @RyanUnderhill in Windows Debug Build. The checker doesn't work with vstest.
4. Fix the post merge pipeline(Mainly for test coverage report).
5. Ignore the compile warning from the Featurizer library code
6. Apply "/utf-8" VC compile flag to our code. Without this, you can't build onnxruntime on Chinese Windows.
7. Remove the SingleUnitTestProject cmake option because it's deprecated more than one year and nobody is using it.
8. Move opaque api tests to onnxruntime_test_all
9. Enable "/W4" on CUDA ep's C++ code(Not the *.cu files), and fix some warnings, add some extra checks.
10. Delete the onnxruntime::test::TestEnvironment class.
11. Add a DLLmain for onnxruntime.dll.
12. Allow dynamic link to libprotobuf
1. Enable warning "4503" # Decorated name length exceeded.
2. Enable warning "4146" # unary minus operator applied to unsigned type.
3. Enable float64 support for the Softmax operator
4. Enable compliance checks for Windows x86 32bits build
5. Use TryBatchParallelFor to replace some fallback code in mlas pooling.cc
6. Fix Android CI pipeline.
Disable DML in Windows GPU CI build for now, because there are some wired model test failure and I don't know how to fix it. Will seek help from WinML team.
* don't run cuda tests if building with tensorrt
* remove unnecessary build options for win trt ci
* refactor win gpu tensorrt ci yml
* --numpy_version=1.17
* update
* update
* azcopy and cuda path
1. Pipeline changes for python 3.8
2. Fix a regression in setup.py which was just introduced in the previous commit.
Please notice, we still haven't made python 3.8 + Windows + CUDA work.
1. Move Win GPU pipeline to VS2019
2. Move C API pipeline to VS 2019
3. Move nuget mklml pipeline to VS 2019
4. Move windows no contrib ops pipeline to VS 2019
Advance commit to 4df80d5865a9d4e97f6d0b9304d4316115a04d9e
Add generated code for the commit before editing.
Import more featurizers.
Rename Automl ops domain to mlfeaturizers.
Rename conditional compilation macro.
Move and rename files getting rid of automl
Rename --use_automl build switch to --use_featurizers
Rename CMake option accordingly. Rename automl CMake targets.
Adjust CI and packaging pipeline switches.
Rename namespace automl to featurizers.
1. temporarily exclude vgg19 test which comsumes too much memory, run out of memory on Upsquared device. Single test pass for vgg19, need furture investigation (#2588)
2. Update docker file to decrease the docker image size
* change c++14 to c++11
* add ld lib path for centos
* enable csharp tests on macos
* fix C API test on MacOS + fix manylinux dotnet install
* fix manylinux dotnet install
* fix lib link
* enabme telemetry
* enable telemetry
* set enable telemetry as default
* for debugging
* remove log and set disable telemetry as default back
* delete private file while testing
* resolve comment: mainly add license header, rename macro and update docs
* rewording in privacy.md
* add centos tests to linux cpu ci pipeline
* Disable failing test
* use centos6 instead of centos7
* change back to centos7
* add dotnet runtime dependency
* fix dotnet runtime dependencies
* install dotnet sdk instead of runtimes
* add more dotnet dependencies
* temporary skip failing test
* ix lib path
* reenable failing test
* add SAS token to download internal test data for nuget pipeline
* update azure endpoint
* fix keyvault download step
* fix variable declaration for secret group
* fix indentation
* fix yaml syntax for variables
* fix setting secrets for script
* fix env synctax
* Fix macos pipeline
* attempt to add secrets to windows download data
* fix mac and win data download
* fix windows data download
* update test data set url and location
1. refactor the pipeline, remove some duplicated code
2. Move Windows_py_GPU_Wheels job to Win-GPU-CUDA10. We'll deprecated the "Win-GPU" pool
3. Delete cpu-nocontribops-esrp-pipeline.yml and cpu-nocontribops-pipeline.yml
4. In Linux nuget jobs, run "make install" before creating the package. So that extra RPAH info will be removed
This change adds a new execution provider powered by [DirectML](https://aka.ms/DirectML).
DirectML is a high-performance, hardware-accelerated DirectX 12 library for machine learning on Windows. DirectML provides GPU acceleration for common machine learning tasks across a broad range of supported hardware and drivers.
The DirectML execution provider is capable of greatly improving evaluation time of models using commodity GPU hardware, without sacrificing broad hardware support or requiring vendor-specific extensions to be installed.
**Note** that the DML EP code was moved verbatim from the existing WindowsAI project, which is why it doesn't yet conform to the onnxruntime coding style. This is something that can be fixed later; we would like to keep formatting/whitespace changes to a minimum for the time being to make it easier to port fixes from WindowsAI to ORT during this transition.
Summary of changes:
* Initial commit of DML EP files under onnxruntime/core/providers/dml
* Add cmake entries for building the DML EP and for pulling down the DirectML redist using nuget
* Add a submodule dependency on the Windows Implementation Library (WIL)
* Add docs under docs/execution_providers/DirectML-ExecutionProvider.md
* Add support for DML EP to provider tests and perf tests
* Add support for DML EP to fns_candy_style_transfer sample
* Add entries to the C ABI for instantiating the DML EP
Enable multi-device test for GPU
* Add build pipeline for TensorRT multi-GPU test
* Add code to disable fp16 test if hardware architecture not supported
* Add option to set the device id in onnx_test_runner for model tests
Remove gsl subodule and replace with a local copy of gsl-lite
Refactor for onnxruntime::make_unique
gsl::span size and index are now size_t
Remove lambda auto argument type detection.
Remove constexpr from fail_fast in gsl due to Linux not being happy.
Comment out std::stream support due to MacOS std lib broken.
Move make_unique into include/core/common so it is accessible for server builds.
Relax requirements for onnxruntime/test/providers/cpu/ml/write_scores_test.cc
due to x86 build.
Add ONNXRUNTIME_ROOT to Server Lib includes so gsl is recognized
1. remove sudo from the cleanup step for Linux so that we don't need the sudo access for vstsagent build user
2. a minor fix in the install_ubuntu.sh to make the image smaller for openvino
* Bump onnx to latest
Update onnx.in.proto with changes for SparseTensor.
* add temp skip tests
* remove passed tests from skip list
* skip more tests for new ops in opset 11
* skip crashing tests
* update handling of new attribute types sparse tensor and sparse tensors
* advance onnx commit and remove skip cpu_flaky_tests
* temporarily skip yolo3 model test due to resize opset10 shape inference regression
* update proto for onnxruntime server
* advance onnx commit further
1. Add openvino GPU nightly build pipeline, this test is running on Intel Up square Edge device. The device are host locally not from Azure VM. We persist a smaller model test data on Edge device.
2. Update the build condition for openvino GPU so it works for GPU_FP32, GPU_FP16
3. add option to install_ubuntu.sh to exclude the package used for nuphar, so that we can save some disk space as the Edge device usually have limited disk space.
Enable Nuphar EP docker build
Revert back to LLVM 6.0.1
Reinstate disabled Softmax tests caused by LLVM 8.0.1
Reinstate Nuphar Python test due to stale sympy version
Increase build timeout of Linux CI
- Fix the Windows end-to-end test in NuGet CI
- Skip the TestModelSerialization, because it is failing on Linux. Must be fixed before API is released for use. Owner is notified.
