* change some function from throw to return status
* move more functions to return status
* move most of the exception to return status
* move logsink on android from CLogSink to AndroidLogSink
* addressed comments
* add type attr to check if the return status is used in compile
Sometimes there is a file named "version.txt" in your CUDA installation dir, but sometimes there isn't one. I couldn't figure out it why, but the latest CUDA 11 on our CI build machines doesn't have this file. As the file is not needed for building onnxruntime, so I removed the check.
Update TransposeMatMul to support scaling of the matrix product by a constant scalar value (analogous to the GEMM alpha parameter). Rename TransposeMatMul to TransposeScaleMatMul.
Fuse MatMul with surrounding Mul/Div with constant scalar into TransposeScaleMatMul.
While investigating an unrelated issue, I noticed that the thread pool may drop tasks when a burst of 1024+ tasks is submitted by a thread from inside the pool. Today, in general, we execute work synchronously in this case. However, there is a bug where work submitted by a thread already inside the pool will be discarded instead of executed. Currently the only scenario where I can see this occurring is when the parallel executor is used with a model in which such a large number of nodes become eligible to run all at once. This PR fixes the underlying issue and adds a test case for burst-submission of work.
Add 'Install ONNX' step to Windows GPU pipeline
Previously it's not a problem because onnxruntime python package explicitly said it depends on ONNX, so ONNX will get installed when we test onnxruntime. However, it was removed in #4073
1. Avoid building ONNX of every history ONNX versions in our CI, it is costly and easy to fail.
2. Run docker command without sudo. Previously the user is not in docker group, now Azure DevOps Service have added it in.
* Revert "Remove docstrigs if __ONNX_NO_DOC_STRINGS" (#4495)
This reverts commit bb4d331fa7bf1fe8d68b1527dda56e4739c80800.
* Bump version to 1.4.0 (#4496)
* Create N-1 threads in intra-op pool, given main thread now active (#4493)
Create N-1 threads in a thread pool when configured with intra-op parallelism of N. This ensures we have N active threads, given that the main thread also runs work. To avoid ambiguity on the value returned, rename ThreadPool::NumThreads method to ThreadPool::DegreeOfParallelism, and make corresponding updates in MLAS and operators.
* Conditionally compile without std::is_trivially_copyable to satisfy old GCC versions. (#4510)
* Adding CUDA arch flags for NVIDIA Jetson
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
* Added Dockerfile for Jetson and instructions to build wheel and image
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
* Removing guess about nvcc location
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
* Restoring pip3 setuptools install order
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
* Updated README with links and notes re NVIDIA Docker runtime
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
* Added mention of nvidia-docker
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
* Addressing code review comments
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
* Addressing code review comments
Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>
Co-authored-by: Tiago Koji Castro Shibata <ticastro@microsoft.com>
Co-authored-by: Dmitri Smirnov <yuslepukhin@users.noreply.github.com>
Co-authored-by: Tim Harris <tiharr@microsoft.com>
Co-authored-by: edgchen1 <18449977+edgchen1@users.noreply.github.com>
* Update Versioning.md
Update documentation to cover latest Windows 10 release (Vb) and the NuGet packages.
* PR feedback.
* readability changes
* spell out Windows ML Availability
* Add ability to retrieve inferred shapes when executing a kernel.
This ability helps Recv to know its output shapes without doing
actual cummunication. Of course, if the output shapes cannot be
inferred, Recv still needs to do communication to get shapes from
Send.
* Avoid communicating shape information when it can be inferred statically
* Replace unordered_map with thread-safe wrapper.
We don't want to have racing condition and undefined behavior
when using parallel executor.y
* Remove cout
* Add missing file
* Address comments
* Check dim_value. -1 means missing
* lock properly
* Address comments (remove thread-safe map)
* Remove poc header
* Replace Stream with DeferredReleaseCPUPtr
* Register ILearningModelSessionOptionsNate interface
* Threading options exposed
* Add interrogator for Session options
* Add test
* Polish test
* PR comments
* Set intra op threads
* Add adapter api to grab intra op threads
* Add adapter test for getting intraop num threads
* Make ILearningModelSessionNative and update winml api test
* Make it required when building engine to set the intraop num threads
* Make test more pretty
* Change naming of idl function
* Revert "Change naming of idl function"
This reverts commit c06916aa5bf94e3bf233ed281e508b935fc8638d.
* PR comment on naming
* Skip the test because it's influenced if it's built with openmp
Co-authored-by: Ryan Lai <ryalai96@gamil.com>
* add dynamic output shape support
* fix bugs associates with scalar inputs
* addressed comments, fixed issue the output buffer size is not correctly set, refactor shaper class
* split the execution logic from nnapi::Model into nnapi::Execution
* update comments for certain scenarios, 1. dynamic output buffer size, 2. ONNX scalar input
* move ctor of nnapi::Execution to public
* remove dependency of external jd-dnnlibrary
* add qlinearadd support
* combine some qlinear ops logics, move some throw into return status
* merge master
* minor bug fixes
* addressed comments
* make dml and onnxruntime system32 only when winml and onnxruntime is loaded from system32
* use __ImageBase as that will not incur the unsupport store api call into GetModuleHandleEx
* remove accidental comment
Co-authored-by: Sheil Kumar <sheilk@microsoft.com>
* added reducesumlogexp gradient
added test
fixed type mismatch when calling cudnnreduce kernel
fixed python frontend to remove redundant states to match pytorch state dict
* minor fix for test dir util
* add pause option for onnx_test_runner
* add flush std to show pause prompt text
Co-authored-by: gwang0000 <62914304+gwang0000@users.noreply.github.com>
* Adding CPU implementation of BroadcastGradientArgs op
Modify to take shape as input instead of tensor
Cleanup
Correct schema
Corrected kernel, added tests, addressed review comments.
Initial change, to add ReduceSumTraining cpu op
cpu support
Initial changes to gradient builder
Non-empty reduction case passing.
Added exception,test for invalid broadcast,addresed review comments.
Initial change, to add ReduceSumTraining cpu op
cpu support
cuda support + more UTs
on comments + UT
no op support for {} axes with new attr - noop_with_empty_axes
Add noop attribute to ReduceSumTraining use
Add testing for no-shape graph, modify AddSub grad builder, logging.:
MulGrad support
Div support
Expand support
Gemm support
MatMul grad change
Transpose Grad change
BiasGeluGrad change.
Fixes after squash
* Remove logging, add specific exception for shape inference error
* fix build
* Review comments
* Review comments
* Fix windows build
Co-authored-by: Ethan Tao <ettao@microsoft.com>
* Adjust indentation of statement, without this fix GCC 7.5 errors
out with:
"this ‘if’ clause does not guard this statement, but the
latter is misleadingly indented as if it were guarded by the ‘if’"
* Add braces around the if-statement for improved clarity.
Co-authored-by: Alberto Magni <alberto.magni@microsoft.com>
