* Reduce the binary size growth from this change. Minimal build grew by 7KB from this checkin.
Firstly simplify the checking logic a little. Same checks are still done - just without using an extra layer of helpers.
The issue being addressed by the original change only applies if you have a graph output where the shape wasn't able to be inferred. e.g. Reshape node with dynamic input causes downstream shapes to be unknown. If that is not the case, MergeShapeInfo in graph.cc would have resolved any differences between a specified output shape and the inferred output shape during Graph::Resolve.
The issue does not apply to the execution frame used by the optimizer as the only time it would create a graph output is if it could constant fold all the way through, so MergeShapeInfo would have handled any difference in that case as well.
Due to these considerations, wiring a logger in at the IExecutionFrame level isn't necessary if VerifyOutputSizes optionally overridden by an implementation that cares.
* Address PR comments
* Adding Output Shape Validation for ORT-CPU execution flow
* Skipping validation check in-case output is not a tensor. Fixed conv_transpose test. Ignoring pad and reduction test
* Comparison b/w signed and un-signed int. Removed const for a primitive variable
* Commented the un-used test function signature
* Removed exception instead logging warning. Because there are lots of ORT tests which are failing because of this validation
* Fixed warning condition and test
* Fixed test and addressed comment on the PR
* Output shape verification will happen only for final output nodes of the model
* Changed variable name from camel case to underscore style
* Enable the tests as the validation failure will now logs warning instead of throwing an exception
* Adding Output Shape Validation for ORT-CPU execution flow
* Resolve merge conflict
* Comparison b/w signed and un-signed int. Removed const for a primitive variable
* Commented the un-used test function signature
* Removed exception instead logging warning. Because there are lots of ORT tests which are failing because of this validation
* Fixed warning condition and test
* Fixed test and addressed comment on the PR
* Output shape verification will happen only for final output nodes of the model
* Changed variable name from camel case to underscore style
* Enable the tests as the validation failure will now logs warning instead of throwing an exception
* Remove duplicate function "GetLogger()"
Remove duplicate function "GetLogger()"
* Fixed typo in method name TestConvTransposeOpInitializer
Fixed typo in method name "TestConvTransposeOpInitializer"
Co-authored-by: Sumit Agarwal <sumitagarwal@microsoft.com>
* initial draft for kernel invoke api
* initial implementation of kernel invoker
* [eager] fix build on Mac
* [eager] increment input name in kernel invoker
* temp fix for type in eager mode
* use global default log manager
* rollback the previous commit since it break linux build
* Revert "rollback the previous commit since it break linux build"
This reverts commit 58c2c3423a.
* Eager Mode: fix linking on macOS
* optimizer_execution_frame: ignore unused lambda capture (model_path)
* fix link issue
* ORTInvoker: set correct input argument tensor element proto types
Do not set a type proto on output arguments to allow ORT to deduce them
* ORTInvoker: create only one logging manager
* Minor fix to set execution provider type correctly. (#7000)
Co-authored-by: Chandru Ramakrishnan <chandru-r@github.com>
* training fix
* support config output ml values in frame, so we can use it to implement inplace update
* Fix range loop error while building. (#7087)
Co-authored-by: Chandru Ramakrishnan <chandru-r@github.com>
* Conditionally link with nsync_cpp if not windows. (#7151)
Co-authored-by: Chandru Ramakrishnan <chandru-r@github.com>
* Fixed initialization order in ORT kernel invoker (#7342)
* Updated constructor of ort_kernel_invoker to take a logger.
* Changed linking order.
* Updated test.
* add inplace ut
* add build option
* Update include/onnxruntime/core/eager/ort_kernel_invoker.h
Co-authored-by: Derek Murray <Derek.Murray@microsoft.com>
* resolve comments in pr
* fix build break;merge from master
* fix build break
Co-authored-by: Cheng Tang <chenta@microsoft.com>
Co-authored-by: Aaron Bockover <abock@microsoft.com>
Co-authored-by: Chandru Ramakrishnan <41447659+chandru-r@users.noreply.github.com>
Co-authored-by: Chandru Ramakrishnan <chandru-r@github.com>
Co-authored-by: Derek Murray <Derek.Murray@microsoft.com>
* Add support for sharing allocators
* Incremental update
* Address some PR comments, add unit tests, add documentation.
* Address PR comments, add tests and some documentation.
* Fix build and test issues
* Remove RegisterAllocator API restoring the OrtAllocator interface changes. Changed docs to reflect this.
Also fixed the orttraining segfault. The segfault was because in the case of training session,
the CPU exec prov is not available at the time the transformers are applied. Changed it to create
a new one.
* Outputs from model execution should always be returned in a newly allocated buffer or an pre-allocated buffer provided in fetches. When an initializer is providing a graph output (e.g. constant folding may result in this) we were returning an OrtValue that pointed to the initializer and not a separately allocated buffer with a copy.
This was wrong as:
- value wasn't returned in a pre-allocated fetch so whilst the value returned was correct, it was returned in the wrong place
- user could alter the data in the initializer via the returned value
* Add unit test with and without pre-allocated fetch.
* Add some extra info around why we're handling this special case.
* Stop proceeding with constant folding if a CPU kernel is not found for a node
* Fix build
* PR feedback
* Fix typo
* Refine
* Remove unnecessary header inclusion
* Refine
* Fix build
* More changes
* More changes
* More changes
* Fix CentOS build
Description:
Disallow overriding an initializer via a graph input if the IR version is < 4. This enforces an implicit assumption that initializers should be treated as constant, and allows constant folding to be done on a model with an older IR version.
Separate constant and overridable initializers so that it's clear which ones constant folding can utilize.
Update Graph to not add all initializers to the graph inputs when the graph is manually created (i.e. not loaded from a GraphProto) and the IR version is >= 4.
Motivation and Context
In order to do constant folding we need to know which initializers can be treated as constant and which are overridable. All initializers were required to have a matching graph input prior to IR version 4, technically making all of them overridable. The intention however was for them to be treated as constants, and this change enforces that intent.
The benefit of doing so is that constant folding will work for models with IR version < 4. The cost is that if someone is actually overriding an initializer they will need to update the IR version of their model to version 4 in order to keep doing so. The belief is that this is a very small subset of usage (e.g. models involving feeding in a truncated sequence) and the cost to update that small subset is warranted by the benefit of constant folding being able to be enabled on all older models without them needing an IR version update.
1. Support the new external data extension in ONNX 1.4 onnx/onnx#678
2. Enable onnxruntime_perf_test in Mac Build
3. move path_lib.h from onnx_test_runner source dir to onnxruntime_framework
4. Enable memory planner for string tensors
5. Make memory planner always enabled, to simplify model loading logic
6. Delete some duplicated code between onnxruntime_perf_test and onnx_test_runner
7. Delete win_getopt_mb lib.
8. Remove the dependency on Pathcch lib, which is only available on Windows 8 and newer.
