* Add deepspeed zero stage 1 poc test with MNIST data to the ORTModule CI pipeline
* Add ORTModule CI pipeline tests for POC and hf BERT classifier with --no-cuda arg
* Fix typo in ORTModule.to()
The `args` and `kwargs` should be expanded in the call to `super(...).to()`.
* Add fixes for multiple CUDA devices.
* Add simple DeepSpeed test script and configuration.
* Fixes for test script and config.
* Add trailing newline.
* Fix formatting for config.
* Set InferenceSession provider options at construction.
* Make the local_rank arg required.
* Convert ORTModule._device to a torch.device() before using its accessors.
* Refactor device handling and fix regressions on BERT fine tuning
Co-authored-by: Derek Murray <demurra@microsoft.com>
Co-authored-by: Thiago Crepaldi <thiago.crepaldi@microsoft.com>
Although forward pass works, this has the limitation of not working for
backward pass due to the lack of intermediate tensors needed for
gradient.
Next step is to export a training graph and split it manually
Move CudaKernel from cuda_common.h to a new separate header, cuda_kernel.h. Update include sites to use cuda_kernel.h instead if they need CudaKernel. Inclusions of cuda_common.h are now more lightweight.
Make corresponding changes for ROCM execution provider code.
Other minor cleanup.
* build for .net5
* only reference cswinrt for .net5
* remove netstandard2.0 references
* upgrade language version
* net5
* remove extra comment closure
* add targetframework
* set target framework
* remove net*
* pep8 errors
* make test project build with .net windows SDK projection
* disable c# builds for non-x64 builds
* fix pep8 errors
* disable for store build
* fix tests
* remove cswinrt and sdk references from package
* bump cswinrt down to 1.0.1
* fix bin path
Co-authored-by: Sheil Kumar <sheilk@microsoft.com>
