* support optimizer opt for deepspeed 0.5.9
* resolve comments
* resolve comments
* FP16_Optimizer Support for more Deepspeed Versions (#12046)
* fp16_optimizer for more ds versions
* change ds version
* bugfix
* fix bug
* Fix unused function warning for decodeMIDR(). (#12069)
Changed from static function defined in header to function declared in header and defined in separate .cc file.
* pin protobuf version to be compatible with onnx (#12132)
Co-authored-by: Ashwini Khade <askhade@microsoft.com@orttrainingdev10.d32nl1ml4oruzj4qz3bqlggovf.px.internal.cloudapp.net>
* RoiAlign CPU EP add warning for max mode with samples != 1 (#12136)
* RoiAlign add warning about incorrect max summation when sample size not 1
* include coreml_provider_factory.h in macos build instead of coreml_ex… (#12138)
include coreml_provider_factory.h in macos build instead of coreml_execution_provider.h
* List 3.10 as supported python version and remove 3.6 (#12141)
list 3.10 as supported python version and remove 3.6
Co-authored-by: Randy Shuai <rashuai@microsoft.com>
* Use updated symbolic_helper.check_training_mode (#11900)
Co-authored-by: Jingyan Wang, Baiju Meswani
* Fix GH issue 12151 by using inverse perms for updating DQ axis attribute (#12158)
* Fix GH issue 12151.
Need to use inverse perms for updating that axis to what is used for transposing the input. This only applies if the DQ node is doing per-axis dequantization.
* fixing positions for beam search gpt2 (#12156)
* fixing positions for beam search gpt2
Co-authored-by: Tianlei Wu <tlwu@microsoft.com>
* remove wrong placed libs (#12201)
* Add file mapping for windows platform. (#12183)
* Add file mapping for windows platform.
* Add unit test for file mapping for windows. Also add an error message for mis-aligned offset
* Add unit test for file mapping for windows. Also add an error message for mis-aligned offset
* Update data type to avoid warnings
* Compitable data type to avoid warnings. Update CreatFileMapping2 condition for winml compiling.
* Add type conversion to avoid warnings for X86 release build.
Co-authored-by: Ting Cao <ticao@microsoft.com>
* Fix bug where onnxruntime_USE_NCCL flag would default to ON (#12195)
Fix bug where onnxruntime_USE_NCCL flag would default to ON, causing ORT to not build properly. New functionality: flag is ON when training is enabled and NCCL is not disabled. Flag is OFF otherwise
Co-authored-by: zhijxu <zhijxu@microsoft.com>
Co-authored-by: zhijxu <zhijxu>
Co-authored-by: Vincent Wang <wangwchpku@outlook.com>
Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com>
Co-authored-by: Ashwini Khade <askhade@microsoft.com>
Co-authored-by: Ashwini Khade <askhade@microsoft.com@orttrainingdev10.d32nl1ml4oruzj4qz3bqlggovf.px.internal.cloudapp.net>
Co-authored-by: Dwayne Robinson <dwayner@microsoft.com>
Co-authored-by: Carson Swope <carsonswope@users.noreply.github.com>
Co-authored-by: Randy Shuai <rashuai@microsoft.com>
Co-authored-by: jingyanwangms <47403504+jingyanwangms@users.noreply.github.com>
Co-authored-by: Scott McKay <skottmckay@gmail.com>
Co-authored-by: Viswanath Boga <44417868+viboga@users.noreply.github.com>
Co-authored-by: leqiao-1 <61653207+leqiao-1@users.noreply.github.com>
Co-authored-by: caoting-dotcom <71617901+caoting-dotcom@users.noreply.github.com>
Co-authored-by: Ting Cao <ticao@microsoft.com>
Co-authored-by: Sean Murray <59740888+seanmurr1@users.noreply.github.com>
* First iteration of making cuda a shared provider.
Separated out shared OpKernel change, so doing this to merge with that change.
* More cuda shared library refactoring
* More cuda shared library refactoring
* More build options tested, converted the training ops over.
* Fix merge breaks
* Fix submodules
* Fix submodules
* Fix submodules
* Fix python
* Fix compile errors
* Duplicate symbol fix
* Test fix for ROCM provider
* Another ROCM test workaround
* ROCM Build Test
* ROCM build fix
* ROCM
* ROCM
* ROCM
* ROCM
* ROCM
* ROCM test
* Reduce header dependencies
* Remove redundant namespace
* Test fix for linux
* Fix linux build
* Fix Eigen build error
* Fix unused parameter warning
* Test link error
* Another linker test
* Linker test
* Linker test
* Another test
* Another build test
* Fix linux link error
* Build test
* Fix control flow ops to use common base class with core code
* Remove extra qualifiers
* Fix template syntax for linux
* Fix cuda memory leak
* Fix pybind
* Test disabling cast
* Cleanup
* Restore cuda in test
* Remove more header dependencies
* Test not adding cuda provider to session
* Make GetProviderInfo_CUDA throw
* No-op cuda provider creation
* Fix some setup issues
* Fix memory cleanup on unload
* Diagnostics
* Don't unload library
* Add diagnostics
* Fix deleting registry at right time.
* Test disabling profiler
* Fix merge break
* Revert profiler change
* Move unloading of shared providers into Environment
* Free more global allocations before library unloads
* Add more diagnostics
* Move unloading back to the OrtEnv as there are multiple Environments created during a session.
Remove some library dependencies for tests.
* Fix more cmake files
* ERROR -> WARNING
* Fix python shutdown
* Test not using dml in pipeline
* Change python version and disable dml
* Update python version
* Test adding unload method for shared providers
* Disable DLL test
* Python test
* Revert "Python test"
This reverts commit c7ec2cfe98.
* Revert "Disable DLL test"
This reverts commit e901cb93aa.
* Revert "Test adding unload method for shared providers"
This reverts commit c427b78799.
* Point to RyanWinGPU
* Revert python version
* Fix id_to_allocator_map
* Another python exit test
* Remove extra debug messages
Try a more clean python shutdown through DllMain
* Revert DllMain idea, it didn't work
* Merge conflicts
* Fix merge with master issues.
* Comments
* Undo edit to file
* Cleanup + new training ops
* Revert yml changes
* Fix another merge error
* ROCM fix
* ROCM fix v2
* Put back Linux hack, it is necessary
* Stupid fixes
* Fix submodule out of sync
* ROCM fix 3
* ROCM 4
* Test java fix
* Fix typos
* Java test on my VM
* Fix build error
* Spotless fix
* Leave temp file around to load properly
* Fix cleanup on exit
* Fix break
* Java comments
* Remove LongformerAttentionBase workaround
* Spotless fix
* Switch yml back to regular build pool
* Revert "Switch yml back to regular build pool"
This reverts commit be35fc2a5a.
* Code review feedback
* Fix errors due to merge
* Spotless fix
* Fix minimal build
* Java fix for non cuda case
* Java fix for CPU build
* Fix Nuphar?
* Fix nuphar 2
* Fix formatting
* Revert "Remove LongformerAttentionBase workaround"
This reverts commit 648679b370.
* Training fix
* Another java fix
* Formatting
* Formatting
* For orttraining
* Last orttraining build fix...
* training fixes
* Fix test provider error
* Missing pass command
* Removed in wrong spot
* Python typo
* Python typos
* Python crash on exit, possibly due to unloading of libraries.
* Remove test_execution_provider from training build
Only enable python atexit on windows
Remove assert on provider library exit
* Still can't unload providers in python, alas.
* Disable Nvtx temporarily
* MPI Kernels for Training
* MPI Kernels part 2
* Patch through INcclService
* Oops, wrong CMakeLists
* Missing namespace
* Fix missing ()
* Move INcclService::GetInstance around to link nicer
* Missing }
* Missing MPI libraries for Cuda
* Add extra GetType functions used by MPI
* Missing Nccl library
* Remove LOGS statements as a test
* Add in a couple more missing GetType methods
* Update comments
* Missed a logging reference in mpi_context.h
* Convert aten_op to shared (due to marge with master)
* Test moving DistributedRunContext instance into shared provider layer
(with purpose error to verify it's being built properly)
* Test passed, now with fix
* Missing static
* Oops, scope DistributedRunContext to just NCCL
* Merge related issues and code review feedback.
* Merge error
* Bump to rel-1.9.1 (#7684)
* Formatting
* Code review feedback for Java build on non Windows
* Remove cupti library dependency from core library
* Test Java pipeline fix
* Linux build fix
* Revert "Linux build fix"
This reverts commit a73a811516.
* Revert "Remove cupti library dependency from core library"
This reverts commit 6a889ee8bf.
* Packaging pipeline fixes to copy cuda shared provider for tensorrt & standard packages
* Add cuda to Tensorrt nuget package
* onnxruntime_common still has a cuda header dependency
Co-authored-by: ashbhandare <ash.bhandare@gmail.com>
1. Fix Nuget package build break caused by #6225
2. Delete Dockerfile.centos_gpu. It is not used anywhere.
3. Fix Linux CUDA 10.2 build error caused by glibc upgrade
* Adding versioned dlls to tar/zip packages
* fix syntax error
* fix version name of dylib
* minor fix in the target
* update pattern for versioned dylib files
* Ensure Linux binaries are built with debug info. Extract debug info out of the main binaries. Strip the main binaries.
* add binutils
* add uname
* add binutils
* remove linux portion
