* bug fix for models not using wrapper
* add test case for no wrapper case
* update test case to use internal learning rate
* fix bug with frozen weight update
* Fixes from investigating issue running BERT-Squad model with larger batch sizes. When the batch size gets large enough the initial run will be successful (no memory pattern in use) but the second will fail to allocate the memory pattern block.
The cause of this failure is that we still have the smaller blocks from the first run allocated, as BFCArena has no logic to free those. This essentially results in 2x the memory being required to run the model.
There was inconsistency in BFCArena::Extend which on one path threw an exception if it couldn't do the allocation, and on another just returned false (resulting in Alloc returning a nullptr). Make the behavior consistent by always throwing if BFCArena fails to find a buffer to return. There are a huge number of places in the code where we assume Alloc returns a valid pointer so throwing will result in more correct behavior as a whole. It's also consistent with what happens when CUDA or the standard library fails to allocate memory.
Next, update ExecutionFrame to check for this failure and not insert a memory block entry if it happens. With the existing code if BFCArena Alloc returned a nullptr we happily inserted that in the blocks, delaying detection of the failure to when we attempted to use the block in AllocateMLValueTensorSelfOwnBufferHelper.
Finally update AllocateMLValueTensorSelfOwnBufferHelper to expect a location may not have a block. A log message will be provided when the block allocation fails so it's not necessary to have more on each individual allocation that would have used the block. Falls through to default behavior of doing a normal allocation.
Dropout op was recently changed to accept a new input named
'training_mode', which is passed in to DropoutGrad automatically.
This PR updates the DropoutGrad schema to accommodate the new input.
Tests were also update to reflect the API change
Co-authored-by: Thiago Crepaldi <thiag.crepaldi@microsoft.com>
* General enhancements/cleanups to test exes
- Support running onnxruntime_perf_test with no output file
- if you're profiling the output file is often unused and can be very large
- Allow failure to override early success if doing multiple runs of a test using running onnx_test_runner
- e.g. if the second run fails that's more important as a final status
- Clarify ownership semantics
- Cleanup naming, line lengths, usage of references for required parameters etc.
* Add ArmNN Execution Provider
Add a new execution provider targeting Arm architecture based on ArmNN.
Validated on NXP i.MX8QM CPU with ResNet50, MobileNetv2 and VGG models.
reviewed-by: mike.caraman@nxp.com
* Minor fixes
- renamed onnxruntime_ARMNN_RELU_USECPU to onnxruntime_ARMNN_RELU_USE_CPU
- fixed acl typo
* remove extra includes. added exception for ArmNN in test
* fix indentation
* Separated the activation implementation from the cpu and fixed the blockage from the endif
Co-authored-by: Andrei-Alexandru <andrei-alexandru.avram@nxp.com>
Needed to change the MissingTrack enum naming due to ort_mutex.h including Windows.h which #defines TRUE and FALSE (via inclusion of fdi_fci_types.h), breaking usage of MissingTrack::TRUE and MissingTrack::FALSE.
* Symbolic shape inference exit on models without onnx opset used
* Temporary fix for ConvTranspose with symbolic input dims
Co-authored-by: Changming Sun <me@sunchangming.com>
* Fixed libjson-c_dev_fix and Updated Readme
* Fix VAD-M naming inconsistency in docs
* Avoid removal of sudo in install_common_deps
* Remove 'sudo' for wget in install_common_deps.sh for dockerfiles
'sudo' is not required, and hinders running script from within
proxy environments. Removing it also makes lines consistent with
each other (there are other wget lines without sudo).
Co-authored-by: gundaarx <mayax.vijayan@intel.com>
1. Fix the nuget cpu pipeline and put code coverage pipeline back.
2. Reduce onnx_test_runner's default logging level from WARNING to ERROR. Because there are too many log messages now.
3. Enlarge the protobuf read buffer size for onnx_test_runner. It was missed from PR #4020.
- Add support for ENABLE_LANGUAGE_INTEROP_OPS in training build which is enabled for nightly builds
- Fix passing of environment variables to `sudo docker run` in build definitions
- Fix setup.py package naming logic
The default branch for the spdlog repository on GitHub recently changed from "master"
to "v1.x", which has a different API for `syslog_sink::syslog_sink()`. This breaks
builds of the server for anyone who has checked out the submodules since that change.
Fixes#4077.
Co-authored-by: Derek Murray <demurra@microsoft.com>
* Add flake8 to Win CI build so it's re-enabled. It was in the static analysis build that is currently disabled so checks are not running.
Fix build.py to be compliant again.
Add prefix to flake8 output so it's (hopefully) easier to identify the errors in build output.
* Add to all builds in Windows CPU CI so they all fail quickly if there's an issue.
* Handle edge case where an implicit input for a subgraph may not get wired in correctly.
Conditions required:
- two or more levels of nested subgraph
- an implicit input from above the bottom two levels is used in both levels of subgraph
- this creates a NodeArg for the implicit input at both levels
- something changes to the first level subgraph to no longer use the implicit input
- could be constant folding, could be partitioning of nodes results in a copy of the implicit input being made to a different device
When that occurs we lose the wiring through to the second level of nested subgraph as there's a NodeArg in the first level but the implicit input is no longer used there. Fix that by doing a final check for outer scope values once we know all the outputs produced by the current graph.
Found by commenting out the CUDA implementations of the control flow nodes and running ssd_mobilenet_300 from the mlperf models.
* Add test case.
* publish mloperatorauthor.h in the nuget
* build dmlep into arm/arm64 builds
* update to not use --use_dml everywhere, but enable custom ops everywhere
* always download directml nuget in winml builds
* always build with dml
* dont build dml for arm
Co-authored-by: Sheil Kumar <sheilk@microsoft.com>
* fixed seg fault when using concrete shape
disable gradient as output
* fix evaluation hang issue for multiple gpu run
* Remove dead code, ORTModel and improve docstrings (#3814)
* Refine ORTTrainer docstring descriptions (#3907)
Description:
Fix 2 edge cases as described here: #3755 (comment)
Create a NodeArg for subgraph inputs even if they have no type. If they are only used as an implicit input to another level of nested subgraph we will not create a NodeArg via any other path
Allow an If output to have no shape. Obscure edge case where a loop carried dependency to a Loop node is passed through a nested If node subgraph (i.e. the Loop subgraph contains an If node with a nested subgraph for the else_branch/then_branch). We can't infer a shape for a loop carried dependency (they may change across iterations), which means we can't infer a shape for the nested If subgraph output either. We have delayed allocation support for If outputs so use that.
Motivation and Context
#3755
* update benchmark_gpt2 to use past state only
* update dynamic axes of input/output tensors
* Remove --use_openmp option since it is default for onnxruntime 1.3 cpu.
* Use same option names as benchmark.py
This class is already part of the protobuf-lite library. We don't need a copy here.
And if we do, we must ensure the signature of every function is exactly the same as the original. However, the upstream code may get changed over time. For example, recently protobuf added a "const" modifier to the FileInputStream::GetErrno(), which may break the build if a user want to use the latest protobuf.
