* fix boost download url (#7843)
* Topo sort the model before saving (#7913)
* checkin toposort
* review comments
* revert and add TODO
* Add shape inference to custom symbolic functions (#7937)
**Description**: As title.
**Motivation and Context**
- PyTorch ONNX exporter heavily depends on ONNX shape inference to export accurate and efficient model. Custom symbolic function exports the op as contrib ops, thus exporter is unable to perform standard onnx shape inference. Models with dynamic shape inputs are affected.
* Fix missing files on linux (#8066)
* [Mobile package] Update required operator config with additional ops for wav2vec2. (#8079)
Add some additional ops to the mobile package that are needed for the wav2vec2 model.
* Add module attribute to ORTModule to support HuggingFace Trainer save_model (#8088)
* Fix input schema extrator for ORTModule (#8098)
* Fix 32bit Android java API crash (#8122)
* Fix 32bit Android java API crash
* fix code formating
* [Mobile package] Update required operator config with additional ops for newer version of Wav2Vec 2. (#8123)
This is an update to https://github.com/microsoft/onnxruntime/pull/8079
The sample application motivating the original update changed to use an updated version of the model. Now, fewer ops are required. This change removes the previously added ops which are no longer needed.
* Add int64 as a required type to ConstantOfShape as it's used by the pytorch converter for Pad. (#8128)
It's also used pointlessly for torch.tensor.repeat (although that usage should always be able to be constant folded).
* Update logic in props.xml to account for shared provider library changes (#8138)
* Ortmodule override torch.manual_seed() (#8131)
* Ortmodule override torch.manual_seed()
* Fix Python Cuda loading issues (#7939)
* Fix mac shared_provider warning (#8153)
Co-authored-by: Guoyu Wang <62914304+gwang-msft@users.noreply.github.com>
Co-authored-by: Ye Wang <52801275+wangyems@users.noreply.github.com>
Co-authored-by: Bowen Bao <bowbao@microsoft.com>
Co-authored-by: Ryan Hill <38674843+RyanUnderhill@users.noreply.github.com>
Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com>
Co-authored-by: baijumeswani <bmeswani@microsoft.com>
Co-authored-by: Thiago Crepaldi <thiago.crepaldi@microsoft.com>
Co-authored-by: Scott McKay <skottmckay@gmail.com>
Co-authored-by: Hariharan Seshadri <shariharan91@gmail.com>
Co-authored-by: Sherlock <baihan.huang@gmail.com>
* Cache initializers and avoid device check ot end of forward (#7905)
* ATenOp Enhancement (#7725)
* config parser, default argument values
* ut
* win build
* maxpool2d
* fix win build
* fix build
* unfold atenop
* Update CMakeLists.txt for openvino EP (#7980)
* Add SoftmaxCrossEntropyLossInternal to Support Dynamic ignore_index Input (#7899)
* add SoftmaxCrossEntropyLossInternal
* bugfix and ut
* fix ut
* fix ut
* support torch1.8.1
* function body for nll_loss_internal
* Override ORTModule named_modules to support extra arg (#7954)
* add missing provider_options.h in packages (#7995)
* consolidate copy binary script for gpu/trt tarball package
* add provider_options.h
* add provider_options.h
* Add cuda provides files (#8002)
* Save module output for backward if needed (#8010)
* Save module output for backward if needed
* Make logic in InsertCastTransformer around forcing a node to fp32 more precise. (#8018)
* Address #7981
Reworked the logic around forcing a node to run on fp32 even if it was supported on fp16.
The github issue had multiple factors. In ORT 1.8 we remove Identity nodes that produce graph outputs as they're not needed. That resulted in a Loop node no longer having output nodes (it produces graph outputs instead), which meant the check in IsSingleInputNodeFloat16Node returned true as there was no longer a downstream Identity node processing fp16 data.
We shouldn't only force a node to fp32 in very specific circumstances, and the changes hopefully check for those more precisely.
* Fix Memory Leak from DlpackToOrtValue (#8029)
* Update DirectML EP changes from DmlDev as of 2021-06-07 (#7987)
* Merged PR 6093117: Fix test_DynamicQuantizedLinear_max_adjusted_expanded by allowing Identity operator to run on non-float inputs
Motivation:
As part of the OnnxConformance Backend tests, DynamicQuantizedLinear_max_adjusted_expanded is failing.
Root Cause:
- The test model has `Identity` operator as one of the node. The input of this node is of non-float data type.
- In DML, `Identity` operator is registered as operator which requires floating input.
- As per `DirectMLSchema.h`, support for non-float input has been added for `Identity` operator in DML but the same has not been reflected in the `OperatorRegistration.cpp`.
Changes:
- Removed all traces of the requiresFloatFormatsForGraph flag from it's definition and usage. This flag was only used for Identity and it's related operator.
- Added null check for the graphOutput nodeArg in GraphDescBuilder.cpp to stop the crash of the test.
Related work items: #33076298
* Merged PR 6103324: Remove usage of non-generic error code (FWP_E_NULL_POINTER)
Motivation:
Addressing Dwayne comment on the previous PR. [Ref: [6093117](https://dev.azure.com/microsoft/WindowsAI/_git/onnxruntime/pullrequest/6093117?discussionId=44292162&path=%2Fonnxruntime%2Fcore%2Fproviders%2Fdml%2FDmlExecutionProvider%2Fsrc%2FGraphPartitioner.cpp)]
Changes:
Inside the DML EP, we should not use some other platform specific error codes. Instead we should a appropriate generic error code.
Related work items: #33076298
Co-authored-by: Sumit Agarwal <sumitagarwal@microsoft.com>
* [js/react_native] Use a mobile ORT instead of a full ORT (#8042)
* Change full ort to mobile ort
* Update Android example to load mobile ort
* Change the format of test models to ort
* update ios to use mobile ort
* revise README
* use onnxruntime-mobile-c CocoaPods in a npm package
* fix PATH addition in windows
should set PATH, not add to the tail the copy of PATH
* Reduce Kernel Optimization (#8067)
* reduce optimization
* bug fix
* add a check
* add ut
* refactor
* add ut cases for keepdims=true
Co-authored-by: baijumeswani <bmeswani@microsoft.com>
Co-authored-by: Vincent Wang <wangwchpku@outlook.com>
Co-authored-by: Changming Sun <chasun@microsoft.com>
Co-authored-by: George Wu <jywu@microsoft.com>
Co-authored-by: Ryan Hill <38674843+RyanUnderhill@users.noreply.github.com>
Co-authored-by: Sherlock <baihan.huang@gmail.com>
Co-authored-by: Scott McKay <skottmckay@gmail.com>
Co-authored-by: sumitsays <sumitagarwal330@gmail.com>
Co-authored-by: Sumit Agarwal <sumitagarwal@microsoft.com>
Co-authored-by: Sunghoon <35605090+hanbitmyths@users.noreply.github.com>
Co-authored-by: iperov <lepersorium@gmail.com>
* Fix bug in Transpose CUDA kernel (#7329)
* Fix permission error for ORTModule lock file (#7814)
* fix topo sort in quant tool (#7833)
* fix topo sort in quant tool
* add unit test and make the topo sort stable
* Relax tol for Conv1D fp16 test (#7844)
* Relax tol for Conv1D fp16 test
Co-authored-by: Sherlock Huang <bahuang@OrtTrainingDev3.af05slrtruoetgaxwwjv5nsq5e.px.internal.cloudapp.net>
* Resolve issue with wrapped ORTModule load_state_dict (#7847)
* Encapsulate children modules inside a ModuleAccessor object to prevent erroneuos iteration over children while loading the state dictionary
* Add named_models, models, apply methods, change ModuleAccessor to ModuleMetadata and modify unit tests
* Change ModuleMetadata module getter logic, raise NotImplementedError for add_modules
* Add comment explaining why overriding _load_from_state_dict method is needed
* fixed bugs in packed mode and enable pack mode tests in ci (#7848)
* fixed bugs in packed mode and enable pack mode tests in ci
* removed unnecessary space
* pr comments
* pr comments
* disable an average pool test
* try disabling another avg pool
* disable more avg pool tests
* disable maxpool tests
* add environment variable to control default training package's local version (#7849)
* [js] update documents (#7852)
* [js] update documents
* escape double quotes
* update operators.md
* resolve comments
* Support bool type for Pad CPU (#7856)
* Initial commit
* update
* nit
* Include ORT C/C++ API headers in the ORT Mobile AAR package (#7858)
* Add header files of ort c/c++ api to aar package
* Move header file selection to cmake based on EP choice
* fix duplicated node name (#7865)
* Clean up CPU kernel definition for opset 13 Pad (#7867)
Co-authored-by: Hariharan Seshadri <shariharan91@gmail.com>
Co-authored-by: Thiago Crepaldi <thiago.crepaldi@microsoft.com>
Co-authored-by: Yufeng Li <liyufeng1987@gmail.com>
Co-authored-by: Sherlock <baihan.huang@gmail.com>
Co-authored-by: Sherlock Huang <bahuang@OrtTrainingDev3.af05slrtruoetgaxwwjv5nsq5e.px.internal.cloudapp.net>
Co-authored-by: baijumeswani <bmeswani@microsoft.com>
Co-authored-by: Tixxx <tix@microsoft.com>
Co-authored-by: liqunfu <liqfu@microsoft.com>
Co-authored-by: Yulong Wang <yulongw@microsoft.com>
Co-authored-by: Guoyu Wang <62914304+gwang-msft@users.noreply.github.com>
Co-authored-by: Tianlei Wu <tlwu@microsoft.com>
* First iteration of making cuda a shared provider.
Separated out shared OpKernel change, so doing this to merge with that change.
* More cuda shared library refactoring
* More cuda shared library refactoring
* More build options tested, converted the training ops over.
* Fix merge breaks
* Fix submodules
* Fix submodules
* Fix submodules
* Fix python
* Fix compile errors
* Duplicate symbol fix
* Test fix for ROCM provider
* Another ROCM test workaround
* ROCM Build Test
* ROCM build fix
* ROCM
* ROCM
* ROCM
* ROCM
* ROCM
* ROCM test
* Reduce header dependencies
* Remove redundant namespace
* Test fix for linux
* Fix linux build
* Fix Eigen build error
* Fix unused parameter warning
* Test link error
* Another linker test
* Linker test
* Linker test
* Another test
* Another build test
* Fix linux link error
* Build test
* Fix control flow ops to use common base class with core code
* Remove extra qualifiers
* Fix template syntax for linux
* Fix cuda memory leak
* Fix pybind
* Test disabling cast
* Cleanup
* Restore cuda in test
* Remove more header dependencies
* Test not adding cuda provider to session
* Make GetProviderInfo_CUDA throw
* No-op cuda provider creation
* Fix some setup issues
* Fix memory cleanup on unload
* Diagnostics
* Don't unload library
* Add diagnostics
* Fix deleting registry at right time.
* Test disabling profiler
* Fix merge break
* Revert profiler change
* Move unloading of shared providers into Environment
* Free more global allocations before library unloads
* Add more diagnostics
* Move unloading back to the OrtEnv as there are multiple Environments created during a session.
Remove some library dependencies for tests.
* Fix more cmake files
* ERROR -> WARNING
* Fix python shutdown
* Test not using dml in pipeline
* Change python version and disable dml
* Update python version
* Test adding unload method for shared providers
* Disable DLL test
* Python test
* Revert "Python test"
This reverts commit c7ec2cfe98.
* Revert "Disable DLL test"
This reverts commit e901cb93aa.
* Revert "Test adding unload method for shared providers"
This reverts commit c427b78799.
* Point to RyanWinGPU
* Revert python version
* Fix id_to_allocator_map
* Another python exit test
* Remove extra debug messages
Try a more clean python shutdown through DllMain
* Revert DllMain idea, it didn't work
* Merge conflicts
* Fix merge with master issues.
* Comments
* Undo edit to file
* Cleanup + new training ops
* Revert yml changes
* Fix another merge error
* ROCM fix
* ROCM fix v2
* Put back Linux hack, it is necessary
* Stupid fixes
* Fix submodule out of sync
* ROCM fix 3
* ROCM 4
* Test java fix
* Fix typos
* Java test on my VM
* Fix build error
* Spotless fix
* Leave temp file around to load properly
* Fix cleanup on exit
* Fix break
* Java comments
* Remove LongformerAttentionBase workaround
* Spotless fix
* Switch yml back to regular build pool
* Revert "Switch yml back to regular build pool"
This reverts commit be35fc2a5a.
* Code review feedback
* Fix errors due to merge
* Spotless fix
* Fix minimal build
* Java fix for non cuda case
* Java fix for CPU build
* Fix Nuphar?
* Fix nuphar 2
* Fix formatting
* Revert "Remove LongformerAttentionBase workaround"
This reverts commit 648679b370.
* Training fix
* Another java fix
* Formatting
* Formatting
* For orttraining
* Last orttraining build fix...
* training fixes
* Fix test provider error
* Missing pass command
* Removed in wrong spot
* Python typo
* Python typos
* Python crash on exit, possibly due to unloading of libraries.
* Remove test_execution_provider from training build
Only enable python atexit on windows
Remove assert on provider library exit
* Still can't unload providers in python, alas.
* Disable Nvtx temporarily
* MPI Kernels for Training
* MPI Kernels part 2
* Patch through INcclService
* Oops, wrong CMakeLists
* Missing namespace
* Fix missing ()
* Move INcclService::GetInstance around to link nicer
* Missing }
* Missing MPI libraries for Cuda
* Add extra GetType functions used by MPI
* Missing Nccl library
* Remove LOGS statements as a test
* Add in a couple more missing GetType methods
* Update comments
* Missed a logging reference in mpi_context.h
* Convert aten_op to shared (due to marge with master)
* Test moving DistributedRunContext instance into shared provider layer
(with purpose error to verify it's being built properly)
* Test passed, now with fix
* Missing static
* Oops, scope DistributedRunContext to just NCCL
* Merge related issues and code review feedback.
* Merge error
* Bump to rel-1.9.1 (#7684)
* Formatting
* Code review feedback for Java build on non Windows
* Remove cupti library dependency from core library
* Test Java pipeline fix
* Linux build fix
* Revert "Linux build fix"
This reverts commit a73a811516.
* Revert "Remove cupti library dependency from core library"
This reverts commit 6a889ee8bf.
* Packaging pipeline fixes to copy cuda shared provider for tensorrt & standard packages
* Add cuda to Tensorrt nuget package
* onnxruntime_common still has a cuda header dependency
Co-authored-by: ashbhandare <ash.bhandare@gmail.com>
1. Move the multi GPU pipeline to CUDA 11.0
2. Exclude the keras2coreml_SimpleRNN_ImageNet model test.
3. Add a test for NV6+CUDA 11.0
BTW, it's known our code doesn’t build with CUDA10.2 + Nvidia T4.
* Install and use conda on ortmodule CI pipelines
* Update build script to install onnxruntime wheel before running unit tests
* Remove python 3.5 from install_python_deps
* Pinning deepspeed version to 0.3.15
* Include ORT format model conversion scripts and infrastructure in ORT python package.
- tweak existing script setup so it can be easily run directly and from the ORT python package
Add config file and readme for Android minimal build package
Update ORT Mobile doco
Disable warning if 'all' optimizations are enabled but NCHWc transformer is excluded (device specific optimizations don't apply in this scenario so the warning is moot).
* Address PR comments
* first attempt rocm training wheel
* modifications needed to python packaging pipeline for Rocm 4.1
* changges to not conflict with cuda
missed stage1 changes
remove package push
add option r to getopt
try again without python install
try again without python install
try again without python install
split pipelines and add back push to remote storage
try on cuda gpu pool
try again
try again
try running without az subscription set
try again on original pipeline
change pool
passing AMD Rocm whl on AMD-GPU pool
split rocm pipeline from cuda pipeline
remove comments
* try adding Rocm tests as well
* try with tests in place
* fix trailing ws
* add training data
* try again as root for tests
* use python3
* typo
* try to map video, render group into container
* try again
* try again
* try to avoid yum error code
* make UID 1001
* try without yum downgrade
* define rocm_version=None
* remove CUDA related comments for Rocm Dockerfile
* Dont pin nightly torch torchvision torchtext versions as they expire (for now nightly is required for Rocm 4.1)
* missed requirements-rocm.txt from last commit
* fix whitespace
