* aten op for inference
* fix build error
* more some code to training only
* remove domain from operator name
* move aten_op_executor ext out from ortmodule
* add pipeline
* add exec mode
* fix script
* fix ut script
* fix test pipeline
* failure test
* rollback
* bugfix
* resolve comments
* enable aten for python build only
* fix win build
* use target_compile_definitions
* support io binding
* turn off aten by default
* fix ut
Co-authored-by: Vincent Wang <weicwang@microsoft.com>
Co-authored-by: zhijxu <zhijxu@microsoft.com>
This reverts commit 1f2c926. Because it makes our packaging pipeline crash
Error message:
[ RUN ] QLinearConvTest.Conv3D_S8S8_Depthwise
Test #1: onnxruntime_test_all ...................Subprocess killed***Exception: 838.24 sec
We haven't successfully reproduced the bug on a real ARM64 hardware. Currently we only saw it showed up with qemu. More investigations are on-going.
* skeleton change
* adam compute kernels
* add rtol/atol for tests
* some clean up
* optional outputs
* more clean up
* add tests
* adamw mode=1 test pass
* clean up tests
* add HF AdamW test cases
* refactor adam test file
* make test pass
* all test pass, fix comments
* rename to adamw
* make test pass again
* fix cpplint
* minor fixes
* fix python lint
* Fix build and tests
* fix builds
* fix windows build
* fix win build
* minor fix
* Refine based on comments
* resolve comments
* formatting
* resolve comments
* add ut
* Implement BitmaskDropout and associated unit tests.
* Implement BitmaskDropoutGrad and associated unit tests.
* Implement Dropout -> BitmaskDropout rewrite rule and associated unit tests.
* Implement (Dropout,DropoutGrad) -> (BitmaskDropout,BitmaskDropoutGrad) rewrite rule.
This commit does not yet include unit tests for this rewrite rule.
This commit also introduces improved documentation for all changes which will be grouped
into this PR.
* bitmask dropout
* fix win build
* bugfix for rocm
* bugfix
* fix code format
* fix ut
* fix build break
* fix ut in win
* resolve comments
* fix ut in trt
* resolve comments
* fix rocm build error
* fix typo
Co-authored-by: Aidan Beggs <aidanbeggs@microsoft.com>
* Fix torch cpp ext build when CPU wheel is installed but GPU card is present
Also there is a minor improvement for ATen operator that allows both
"::op" and "aten::op" name for operators
* Fix flake8 false positive
* [UPDATE] update amd ci pipeline 2 rocm5.1.1
* [FIX] json format error
* [ERROR] disable unit tests
* [FIX] ucx error
* [FIX] cmake version
* [FIX] units test
* support ort device tensor in ort module inference
* fallback aten equal to cpu; add ortmodule inference test case
* fix python format
Co-authored-by: Cheng Tang <chenta@microsoft.com@orttrainingdev9.d32nl1ml4oruzj4qz3bqlggovf.px.internal.cloudapp.net>
This reverts commit 4983d6e5d6. We can't destroy OrtEnv through python's atexit function, because at that time there might be many other ORT python objects alive.
* initial fix
* refactor the function handle
* update the implementation
* fix linux build break
* fix training build
* fix minmal build
* fix gradient checker
* deprecate the local function members in graph. host it in model
* fix changming's comments
* fix comments about inlined containers
* fix a missed inlined container
* fix training build
* avoid const for std string_view
Co-authored-by: Cheng Tang <chenta@microsoft.com>
* add aten export for max, max.dim
* rewrite grad of max (no dim); add cases for min
* update UT cases
* mod sym shape infer
* resolve comments: shape infer, add comments, etc.
* add test for torch.max of two tensors
* resolve peng's comments: keepdim; test case
* correct python format
* fix recently introduced lint error
Description: Set black's target version to be py37 - py310
Motivation and Context
Black by default targets its format for py3.10. Since our project supports python 3.7, we need to target version to all the python versions supported.
Re-ran black. 13 files reformatted.
Description: Format all python files under onnxruntime with black and isort.
After checking in, we can use .git-blame-ignore-revs to ignore the formatting PR in git blame.
#11315, #11316
Prior to this, certain shape and type errors were surfaced only when
the model was using the latest known op set version.
Providing users an explicit option allows for better testing of code
that produces models, which includes unit tests within this repo and
other repos such as the TF-ONNX and PT-ONNX converters.
Remove the previous behavior which seems quite counter-intuitive:
an otherwise identical model with a later op set version should be treated
identically in this regard.
The option defaults to false to avoid causing errors for users that
rely on the previous permissive behavior.
Turned on the strict enforcement by default in OpTester, which revealed a few
disagreements between ORT and ONNX on what the correct output shape should
be.
Fix shape inference bug in ReduceSumTraining with noop_with_empty_axes=1
which was revealed.
Fix TensorOpTest.Unsqueeze_scalar, which was testing negative axes on an
op set version where the op did not actually support negative axes.
Fixes#9506.
I disabled some tests temporarily. I will move them to a separated executable file in another PR.
In the future, I want to combine onnxruntime::Environment and OrtEnv classes. Now we have 3 env classes, it is too confusing:
1. onnxruntime::Env
2. onnxruntime::Environment
3. OrtEnv
Our python binding uses onnxruntime::Environment, while all other language bindings use OrtEnv. So python doesn't unload EPs but the others do. It's better to make them consistent.
Please note even I added the call, currently the unload function still is a no-op on Linux. So, currently on Windows we must unload the EPs while on Linux we must not do it.
* Improve transfered time from ort to torch
* Use static_cast
* fix call to Python API for python <= 3.8
* investigation
* fix ref counts
* disable import if no training
* one function to convert multiple ortvalues
* add proto_type
* enforce dlpack->deleter to be not null
* fix _ortvalues_to_torch_tensor for eager mode
* rename proto_type into element_type in the Python API
* conversion from ort to torch 2x times faster
* fix conversion of list of OrtValue
* replace has_bool_tensor by bool_tensor_indices
* introduce _ortvalues_to_torch_tensor_list
* use _ortvalues_to_torch_tensor_list for cache
* fix ambiguity between c and python classes
Co-authored-by: xavier dupré <xavier.dupre@gmail.com>
Co-authored-by: Thiago Crepaldi <thiago.crepaldi@microsoft.com>
* adding fill scalar for torch ones direct initialization on device and adding test case for it
* using ConstantOfShape to for implementing fill Scalar in atenops
* adding case for handling at::Tensor attribute
* handling the at::Tensor type for ConstantOfShape
* handling the at::Tensor type for ConstantOfShape with attr type
* handling the at::Tensor type case
* converting the data to tensor in case of aten tensor mapping is needed
* handling aten tensor case
* handling aten tensor case and reversing the string case
* changing type of scalar
* improve NonZero
* fix megatron_fp16 optimzier, fix the doc
* multi_tensor_applier
* resolve comment
* fix building warning
* fix build error when enabling training and use tensorrt
* Adding optimization step and step parameter to the ORTTrainer constructor
* Added ORTTrainerOptions for optimization step
* Adding Train Step Info Settings to State Dictionary
* Adding train step info key
* Updating comments
* Reverting changes
* Updating test case for new state dict entry train_step_info
* Update DropoutGrad function to support bfloat16
* Eliminate dead comments
* Set opset version for testcase
Signed-off-by: Ganesan Ramalingam <grama@microsoft.com>
* Update to new builder
Signed-off-by: Ganesan Ramalingam <grama@microsoft.com>
* restore random states after export_model
* move get/set_random_states inside _export_model
* add comments for random state save/restore
* add unit test for random state check
* resolve comments
* fix error
Add runtime optimization support to ONNX -> ORT format conversion script.
Replace `--optimization_level`, `--use_nnapi`, and `--use_coreml` with a new `--optimization_style` option.
* creating a test for printing ort tensor
* modifying comment for error case
* Using Output Grabber to assert the print output
* modifying the print ort test
* removing comments
* removing sys import
