* Use list comprehensions instead of list appends where possible
* Add OrtValueVector class as an opaque object in pybind
* Add dlpack methods to the OrtValueVector pybind class
* First iteration of making cuda a shared provider.
Separated out shared OpKernel change, so doing this to merge with that change.
* More cuda shared library refactoring
* More cuda shared library refactoring
* More build options tested, converted the training ops over.
* Fix merge breaks
* Fix submodules
* Fix submodules
* Fix submodules
* Fix python
* Fix compile errors
* Duplicate symbol fix
* Test fix for ROCM provider
* Another ROCM test workaround
* ROCM Build Test
* ROCM build fix
* ROCM
* ROCM
* ROCM
* ROCM
* ROCM
* ROCM test
* Reduce header dependencies
* Remove redundant namespace
* Test fix for linux
* Fix linux build
* Fix Eigen build error
* Fix unused parameter warning
* Test link error
* Another linker test
* Linker test
* Linker test
* Another test
* Another build test
* Fix linux link error
* Build test
* Fix control flow ops to use common base class with core code
* Remove extra qualifiers
* Fix template syntax for linux
* Fix cuda memory leak
* Fix pybind
* Test disabling cast
* Cleanup
* Restore cuda in test
* Remove more header dependencies
* Test not adding cuda provider to session
* Make GetProviderInfo_CUDA throw
* No-op cuda provider creation
* Fix some setup issues
* Fix memory cleanup on unload
* Diagnostics
* Don't unload library
* Add diagnostics
* Fix deleting registry at right time.
* Test disabling profiler
* Fix merge break
* Revert profiler change
* Move unloading of shared providers into Environment
* Free more global allocations before library unloads
* Add more diagnostics
* Move unloading back to the OrtEnv as there are multiple Environments created during a session.
Remove some library dependencies for tests.
* Fix more cmake files
* ERROR -> WARNING
* Fix python shutdown
* Test not using dml in pipeline
* Change python version and disable dml
* Update python version
* Test adding unload method for shared providers
* Disable DLL test
* Python test
* Revert "Python test"
This reverts commit c7ec2cfe98.
* Revert "Disable DLL test"
This reverts commit e901cb93aa.
* Revert "Test adding unload method for shared providers"
This reverts commit c427b78799.
* Point to RyanWinGPU
* Revert python version
* Fix id_to_allocator_map
* Another python exit test
* Remove extra debug messages
Try a more clean python shutdown through DllMain
* Revert DllMain idea, it didn't work
* Merge conflicts
* Fix merge with master issues.
* Comments
* Undo edit to file
* Cleanup + new training ops
* Revert yml changes
* Fix another merge error
* ROCM fix
* ROCM fix v2
* Put back Linux hack, it is necessary
* Stupid fixes
* Fix submodule out of sync
* ROCM fix 3
* ROCM 4
* Test java fix
* Fix typos
* Java test on my VM
* Fix build error
* Spotless fix
* Leave temp file around to load properly
* Fix cleanup on exit
* Fix break
* Java comments
* Remove LongformerAttentionBase workaround
* Spotless fix
* Switch yml back to regular build pool
* Revert "Switch yml back to regular build pool"
This reverts commit be35fc2a5a.
* Code review feedback
* Fix errors due to merge
* Spotless fix
* Fix minimal build
* Java fix for non cuda case
* Java fix for CPU build
* Fix Nuphar?
* Fix nuphar 2
* Fix formatting
* Revert "Remove LongformerAttentionBase workaround"
This reverts commit 648679b370.
* Training fix
* Another java fix
* Formatting
* Formatting
* For orttraining
* Last orttraining build fix...
* training fixes
* Fix test provider error
* Missing pass command
* Removed in wrong spot
* Python typo
* Python typos
* Python crash on exit, possibly due to unloading of libraries.
* Remove test_execution_provider from training build
Only enable python atexit on windows
Remove assert on provider library exit
* Still can't unload providers in python, alas.
* Disable Nvtx temporarily
* MPI Kernels for Training
* MPI Kernels part 2
* Patch through INcclService
* Oops, wrong CMakeLists
* Missing namespace
* Fix missing ()
* Move INcclService::GetInstance around to link nicer
* Missing }
* Missing MPI libraries for Cuda
* Add extra GetType functions used by MPI
* Missing Nccl library
* Remove LOGS statements as a test
* Add in a couple more missing GetType methods
* Update comments
* Missed a logging reference in mpi_context.h
* Convert aten_op to shared (due to marge with master)
* Test moving DistributedRunContext instance into shared provider layer
(with purpose error to verify it's being built properly)
* Test passed, now with fix
* Missing static
* Oops, scope DistributedRunContext to just NCCL
* Merge related issues and code review feedback.
* Merge error
* Bump to rel-1.9.1 (#7684)
* Formatting
* Code review feedback for Java build on non Windows
* Remove cupti library dependency from core library
* Test Java pipeline fix
* Linux build fix
* Revert "Linux build fix"
This reverts commit a73a811516.
* Revert "Remove cupti library dependency from core library"
This reverts commit 6a889ee8bf.
* Packaging pipeline fixes to copy cuda shared provider for tensorrt & standard packages
* Add cuda to Tensorrt nuget package
* onnxruntime_common still has a cuda header dependency
Co-authored-by: ashbhandare <ash.bhandare@gmail.com>
Co-authored-by: Chen Fu <fuchen@microsoft.com>
Description:
This change add google benchmark git repo as a submodule in onnxruntime repo.
Motivation and Context
Currently we have benchmarking code that depends on google benchmark. The version we are using has cross compilation issues for ARM CPUs. Recent changes in Google benchmark fixed these issues.
Another problem is that we now rely on ONNX to pull in Google benchmark, an indirect dependency. Updating ONNX involves complex steps and rightly so. However, updating Google benchmark dependency should not be hindered by these processes.
* Merge set custom allocator to master
* Add documentation for the new API.
Reset global env in testCustomArenaAllocator so won't have a registered allocator of type arena (from previous test)
* Add a session option config that will allow to disable loading model with initializers that have an external data (+test it).
* Add the model used for the test and its external initializers data
* Change the session config option that disable external initializers to a build option.
* Addressing PR comments
1. Move the multi GPU pipeline to CUDA 11.0
2. Exclude the keras2coreml_SimpleRNN_ImageNet model test.
3. Add a test for NV6+CUDA 11.0
BTW, it's known our code doesn’t build with CUDA10.2 + Nvidia T4.
In some environments you would like to link to the shared libprotobuf from the system. This is fine to use if you have full control over all protobuf users in your environment.
This need for the existence of this option is already mentioned as a comment in CMakeLists.txt, this adds it now.
* GPT2 with one step search tutorial
* remove quantization section
Co-authored-by: Xiaoyu Liu <xiaoyu@xiaoyu-VM.z4vh1dzj5eoevgybsksdpz2izh.jx.internal.cloudapp.net>
I saw a test timeout in our nodejs packaging pipeline. I'm not sure if it is because it ran slower than before or it's a deadlock issue. Increasing the timeout will be helpful for investigating such issues.
* Add FBGEMM submodule
* Add fbgemm based per-channel quantization
* Add missing logic for pre-layernorm transformer model fusion
* add support for structured pruning architecture -fastformers
* Fix windows build
* Add a default behavior when head_size is not present for the backward compatibility
* Remove FBGEMM and default to tensor-wise quantization, column-wise quantization will be enabled later
* Fixed some unit test errors
* Fix windows compile error and unit test errors
* delete the option removed from the upstream
* Addresses review comments and fixes a merge error
* Remove commented out code
* add non-zero zp support
* support A and B scale with any dimensions
* fix build breaks
* fix warning in MSVC
* Fix bug for not checking original float value names when treat it as not existing.
* Clean up head size
* Clean up python tools
* Enable per column quantization
* fix quant weight cleanup bug
* A few code clean up
* Some code clean-up
* Some code clean-up
* Change option name
* update default value
* Rename option and parameter names
* Missing argument name change
* Add tests for quantization options for attention and matmul
Co-authored-by: Yufeng Li <liyufeng1987@gmail.com>
Co-authored-by: Lei Zhang <zhang.huanning@hotmail.com>
* add all session options and run options in C API except AddInitializer and AddFreeDimensionOverride
* remove unnecessary comment
* change extra session and run options to object notation
* resolve comments
* use an optional chaining for options
* resolve comments
