- Improves symbolic shape inference in following ways:
1. Extend suggested merge to map to literals with --auto_merge. For example, MatMul of ['ax1', 'ax2'] x [128, 256] would now map 'ax2' to 128
2. Add --int_max option to simplify computations like Min(100000, 'dim') to be 'dim'. This helps ops like Slice to generate correct shape, i.e. start=0, end=Min(100000, dim - 2) on dim. It was previously treated as equal, since sympy cannot determine Min(100000, dim - 2) < dim.
- Fix a bug in create_shared script on Windows, that AOT dll is not generated because of failure in link, when there are too many obj files
- Fix a bug for Split since TOPI does not support split on symbolic dimension.
- Some build warning fixes for NupharEP.
* dump cuda tensor
* move data_type definition
* Dump cuda tensors for cuda build only.
Output tensor location (if it is not in CPU or pinned)
* update for cuda build
* Update for code review feedback
* update for CR feedback
* use data transfer manager for tensor copy
* Guard unused parameter
Guard unused parameter for Linux Arm and other cases.
* Add ACL (Arm Compute Library) execution provider
Add a new execution provider targeting Arm architecture based on Arm Compute Library.
Validated on NXP i.MX8QM CPU with ResNet50, MobileNetv2 and VGG models.
All unit tests are passing.
Comparative performance improvements for ResNet50v1 model obtained with
onnxruntime_perf_test:
A72 2xA72 A53 4xA53
ACL vs CPU 16% 9% 21% 13%
Usage documentation available in ACL-ExecutionProvider.
* Fix eigen unused parameter
Fix eigen unused parameter error for Arm cross-compilation.
* implement cuda topk
* implement heap
* add type support
* refactor interface
* add support for sorting by index
* add test case
* use cub device radix sort
* register for opset 9 and 10
* add opset 9/10 delaration
* refactor code
* refactor code
* fix comment
* fix comment
* switch to scratched mem
* [Nuphar EP] performance improvements
1. Add new ops: Shape, Expand
2. Add support for steps in Slice
3. Simplify Gather
4. Always inline alias nodes
5. Transpose nodes with inner loop being symbolic falls back to CPU provider when vectorization is not possible
6. Add opt_inproj option to model_editor to extract MatMuls inside Scan for input projection to outside
* Add node and op type info to error message if there's a type or shape inferencing exception thrown by the ONNX checker.
* Fix line break from auto format
* Remove unused param from unit test code.
Improve code readability and performance. (#2257)
Remove one time checks from loops.
Move out GetType<>() calls from loop as they
go through local function statics.
Get rid of index calculations from input and output
so we can simlpy advance ptrs and potentially do better pre-fetch.
Improve code readability.
