* Direct use python numpy array's memory if already contiguous. This
could greatly improve performance for session with large input,
like big image 1920x1080 fastrcnn, 30~40% speed up could be achieved.
* Add test case enforce contiguous/non-contiguos numpy array as inputs.
* onnxrt server with OVEP
* onnxrt server with OVEP
* Update Dockerfile.server.openvino
* onnxrt server OVEP fix reviews
* onnxrt server OVEP fix reviews
* Update the CUDA Where implementation broadcasting logic to handle a dim with value of 0.
Add unit test
Also add unit test for unary op with dim value of 0
* Exclude ngraph from Where test with 0 dim.
1. refactor the pipeline, remove some duplicated code
2. Move Windows_py_GPU_Wheels job to Win-GPU-CUDA10. We'll deprecated the "Win-GPU" pool
3. Delete cpu-nocontribops-esrp-pipeline.yml and cpu-nocontribops-pipeline.yml
4. In Linux nuget jobs, run "make install" before creating the package. So that extra RPAH info will be removed
* Remove unused initializer from GraphProto as well as name_to_initial_tensor_ in CleanupUnusedInitializers.
This means initializers that have been replaced during graph optimizations are not left in the GraphProto when we save an optimized model.
* Handle edge case where a model has an unused initializer with matching graph input by also removing the graph input.
* Use non-const iterators in std::find_if calls to make centos build happy.
Description: Describe your changes.
Make elementwise op run 4 items per thread
unroll for loop to leverage ILP
remove unnessary N==0 check inside elementwise GPU kernel
Motivation and Context
Why is this change required? What problem does it solve?
It can improve the performance of GPU elementwise ops. ~2% performance gain on popular NLP bert model.
If it fixes an open issue, please link to the issue here.
thread_local/global/static destruction order depends on implementation details of compilers and OS. The bug happens when thread_local is already out of scope while static EP being destructed, thus causing access violation in EP's destructor when accessing thread_local.
The fix is to maintain ownership inside EP with a mapping from tid to ThreadLocalContext, to avoid accessing thread_local in EP's destructor. This way, no matter what the destruction order is, no access violation would be triggered.
* Add logic to try and flatten inner dimensions being copied by Slice and do a block copy if they can be.
Do a block copy for just the inner most dimension where possible (applies even if we don't flatten inner dimensions).
- Improves symbolic shape inference in following ways:
1. Extend suggested merge to map to literals with --auto_merge. For example, MatMul of ['ax1', 'ax2'] x [128, 256] would now map 'ax2' to 128
2. Add --int_max option to simplify computations like Min(100000, 'dim') to be 'dim'. This helps ops like Slice to generate correct shape, i.e. start=0, end=Min(100000, dim - 2) on dim. It was previously treated as equal, since sympy cannot determine Min(100000, dim - 2) < dim.
- Fix a bug in create_shared script on Windows, that AOT dll is not generated because of failure in link, when there are too many obj files
- Fix a bug for Split since TOPI does not support split on symbolic dimension.
- Some build warning fixes for NupharEP.
* dump cuda tensor
* move data_type definition
* Dump cuda tensors for cuda build only.
Output tensor location (if it is not in CPU or pinned)
* update for cuda build
* Update for code review feedback
* update for CR feedback
* use data transfer manager for tensor copy
* Guard unused parameter
Guard unused parameter for Linux Arm and other cases.
* Add ACL (Arm Compute Library) execution provider
Add a new execution provider targeting Arm architecture based on Arm Compute Library.
Validated on NXP i.MX8QM CPU with ResNet50, MobileNetv2 and VGG models.
All unit tests are passing.
Comparative performance improvements for ResNet50v1 model obtained with
onnxruntime_perf_test:
A72 2xA72 A53 4xA53
ACL vs CPU 16% 9% 21% 13%
Usage documentation available in ACL-ExecutionProvider.
* Fix eigen unused parameter
Fix eigen unused parameter error for Arm cross-compilation.
* implement cuda topk
* implement heap
* add type support
* refactor interface
* add support for sorting by index
* add test case
* use cub device radix sort
* register for opset 9 and 10
* add opset 9/10 delaration
* refactor code
* refactor code
* fix comment
* fix comment
* switch to scratched mem
* [Nuphar EP] performance improvements
1. Add new ops: Shape, Expand
2. Add support for steps in Slice
3. Simplify Gather
4. Always inline alias nodes
5. Transpose nodes with inner loop being symbolic falls back to CPU provider when vectorization is not possible
6. Add opt_inproj option to model_editor to extract MatMuls inside Scan for input projection to outside
