* Allow specific optimizers to be disabled.
- replace unused ability to specify just the optimizers to run
- never used so not needed
Allow the disabled list to be specified via the python bindings
- expected usage is internal, so using kwargs for that so as not to pollute the documentation with stuff no user is likely to need
Update the ORT format model conversion script to disable NCHWc transformer when level is 'all'
- currently there aren't any known use cases where we'd want the NCHWc transformations to run as they create a device specific model and aren't used on ARM
- the ORT format model is not expected to be generated on the target device (e.g. generate on Windows/Linux/macOS to deploy to Android/iOS so there's a good chance we'd generate a useless/invalid model
- default to 'all' as ARM and MLAS prefer NHWC and the NHWC transformer runs at that level
* Add matching changes to optimizer generation in training code
* Fix Transpose and MatMul fusion code to check the input datatypes as FusedMatMul only supports floating point datatypes.
* Added testcases to make sure that the int32/int64 datatypes prevent Transport-MatMul fusion.
* Add function body to SoftmaxGrad schema
* Add type context and cleanup
* Add test case with symbolic dimensions
* Add opset specification to function
* handle opset dependence
* Exclude from minimal build
* Promote BiasDropout from orttraining to onnxruntime
Co-authored-by: Sherlock Huang <bahuang@OrtTrainingDev3.af05slrtruoetgaxwwjv5nsq5e.px.internal.cloudapp.net>
* Liqun/ort module perf1 (#6806)
add mysql script to log perf data
Co-authored-by: liqun <liqun@OrtTrainingDev4.af05slrtruoetgaxwwjv5nsq5e.px.internal.cloudapp.net>
* Resolve HTTP Error 503: Service Unavailable for MNIST dataset (#6989)
* Reduce logging for ORTModule for the end user (#6982)
* Support none types in forward output (#7001)
* Missed test case for none type output (#7014)
* save iobinding to ctx
* save run_options to ctx
* remove debug tests
* PR comments and clean up
* add RunStateInfo
* remove whitespace edits
* PR comments
* remove test changes
* fix test failure
* Fit unit test test_nesting_forward_backward_calls
Co-authored-by: liqunfu <liqfu@microsoft.com>
Co-authored-by: baijumeswani <bmeswani@microsoft.com>
Co-authored-by: Jingyan Wang <jingywa@OrtTrainingDev3.af05slrtruoetgaxwwjv5nsq5e.px.internal.cloudapp.net>
* Add missing Python dependencies for training
cerberus - option parsing
h5py - checkpoint
onnx - model proto
packaging/sympy - symbolic shape inference
* Separate requirements.txt for inference and training Python packages.
Enable type reduction for Shrink, Sign, SplitToSequence CPU kernels.
Some other type reduction changes including refactoring to specify element types in a single place.
Add types support for DenseToSparse and SparseToDense conversions
Address the case of empty sparse values and indicies when the initializer does
not contain any NNZ.
Add sparsify script.
Update the kernel def hashing in ORT format models. The new hashing logic ignores the ordering of type constraint types.
This is a backward compatibility breaking change, but we don't guarantee backward compatibility yet.
