Resize is spec'd to ignore the "roi" tensor in certain modes. For some reason, converters are specifying an arbitrary value for this tensor, even though it's optional.
This makes the graph partitioner skip a check for empty shape dimensions for tensors such as this, which the DML kernel registers as consuming as CPU inputs. Otherwise, the node is not included in DML graph partitions, because the DML graph doesn't handle empty dimensions.
Related work items: #32221164
* Liqun/ort module perf1 (#6806)
add mysql script to log perf data
Co-authored-by: liqun <liqun@OrtTrainingDev4.af05slrtruoetgaxwwjv5nsq5e.px.internal.cloudapp.net>
* Resolve HTTP Error 503: Service Unavailable for MNIST dataset (#6989)
* Reduce logging for ORTModule for the end user (#6982)
* Support none types in forward output (#7001)
* Missed test case for none type output (#7014)
* save iobinding to ctx
* save run_options to ctx
* remove debug tests
* PR comments and clean up
* add RunStateInfo
* remove whitespace edits
* PR comments
* remove test changes
* fix test failure
* Fit unit test test_nesting_forward_backward_calls
Co-authored-by: liqunfu <liqfu@microsoft.com>
Co-authored-by: baijumeswani <bmeswani@microsoft.com>
Co-authored-by: Jingyan Wang <jingywa@OrtTrainingDev3.af05slrtruoetgaxwwjv5nsq5e.px.internal.cloudapp.net>
* Add missing Python dependencies for training
cerberus - option parsing
h5py - checkpoint
onnx - model proto
packaging/sympy - symbolic shape inference
* Separate requirements.txt for inference and training Python packages.
Enable type reduction for Shrink, Sign, SplitToSequence CPU kernels.
Some other type reduction changes including refactoring to specify element types in a single place.
Add types support for DenseToSparse and SparseToDense conversions
Address the case of empty sparse values and indicies when the initializer does
not contain any NNZ.
Add sparsify script.
Update the kernel def hashing in ORT format models. The new hashing logic ignores the ordering of type constraint types.
This is a backward compatibility breaking change, but we don't guarantee backward compatibility yet.
* Code refactor
* Modify code to tackle OOM when calibrating on larget dataset
* Fix mismatch issue when setting keepdims on ReduceMin/ReduceMax
* Add COCO val 2017 annotation
* Fix mismatch issue when setting keepdims on ReduceMin/ReduceMax
* Fix bug of "No module named:onnxruntime.quantization.CalTableFlatBuffers"
* Check and install flatbuffers module
* Add script to donwload coco dataset image and refactor example
* Fix bug of "No module
named:onnxruntime.quantization.CalTableFlatBuffers"
* Add CalTableFaltBuffers as module
* Remove annotation, user can download by themselves.
* Uncommet code
* Add back instances_val2017.json
* Make sure flatbuffers installed when ORT is installed
* Refactor code to call coco api
* Enable FP16 for example
* Added new Transpose+Cast+MatMul => Cast+FusedMatMul test scenarios.
* The Cast node may feed more than one node.
* Transpose node may feed multiple nodes and still may be fused with MatMul nodes.
A model from one of our partners regressed with a failure to evaluate due to the addition of strided 64-bit emulation in the DML EP for the Cast operator. Specifically, the model uses a Cast from int32 to int64 to produce the input shape to a Reshape node. When supplied with a shape dimension of -1 (int32 0xffffffff), the strided emulation in Cast ends up producing an int64 result of 0x00000000ffffffff. This is then fed into the Reshape operator, where it produces an incorrect tensor shape and a failure during evaluation.
Generally speaking we assume that using strided 64-bit emulation is safe if a node's inputs came from the DML EP itself. This isn't true in the general case for Cast, however - casting negative signed values can and will produce incorrect outputs with strided emulation.
After this change, Cast nodes with 64-bit types will fall back to CPU unless running on a GPU that native supports 64-bit datatypes.
Related work items: #31768166
* Liqun/ort module perf1 (#6806)
add mysql script to log perf data
Co-authored-by: liqun <liqun@OrtTrainingDev4.af05slrtruoetgaxwwjv5nsq5e.px.internal.cloudapp.net>
* Resolve HTTP Error 503: Service Unavailable for MNIST dataset (#6989)
* Reduce logging for ORTModule for the end user (#6982)
* Support none types in forward output (#7001)
* Missed test case for none type output (#7014)
* Fix code style according to autopep8
Co-authored-by: liqunfu <liqfu@microsoft.com>
Co-authored-by: baijumeswani <bmeswani@microsoft.com>
Add ARM64X implementation libs, to be forwarded to by the ARM64X lib.
From Ben Niu:
For system dlls that are built outside of windows repo and ingested through vpacks or binary check-ins, we always start by trying to port them to ARM64X. However, due to immature support for ARM64X build from Visual Studio 2019, it could be quite uphill to port dlls to ARM64X.
When that happens, we have an alternative without porting the dll to ARM64X. The alternative solution is, we build an ARM64X pure forwarder from windows repo, for example, onnxruntime.dll. That forwarder does nothing but forwards all the ARM64 API calls to a native ARM64 onnxruntime_arm64.dll, and all the x64 APIs to native x64 onnxruntime_amd64.dll. Please see here for an example: 29ae6ca516
At load time, applications still loads the ARM64X forwarder onnxruntime.dll. In an ARM64 process, that forwarder dll will further load the native ARM64 onnxruntime_arm64.dll; otherwise, the x64 onnxruntime_amd64.dll will be loaded, both the ARM64 and x64 dlls are happy.
The onnxruntime_arm64.dll and onnxruntime_amd64.dll are essentially aliases of their native counterparts, but we cannot directly rename existing native dlls in windows build. The reason is about PDB binplacing. If you simply rename a dll, the PDB name embedded in the dll is still unchanged. So you can imagine that if we just rename the native dlls in ARM64 windows build, there will be two renamed native dlls, onnxruntime_arm64.dll and onnxruntime_amd64.dll, sharing the same PDB name onnxruntime.pdb. When binplacing happens (basically moving dll and pdb from os\obj to os\bin), one PDB will overwrite the other. As a result, we either lose the PDB for the ARM64 dll, or the x64 dll.
That’s why we are asking to change the build pipeline to execute the link commands two extra times to produce onnxruntime_arm64/amd64.dll with different pdb names. You don’t need to do the compilation twice, but just the link. See here for an example: https://microsoft.visualstudio.com/DefaultCollection/Xbox/_git/Xbox.ShaderCompiler.WinTools/pullrequest/5291717
Related work items: #31925159
Change int32_t->ptrdiff_t when interacting with the threadpool.
Migrate more code from MlasMaskMoveAvx->MlasMaskMoveTableAvx.
Update more code to use FUNCTION_ENTRY macro.
