* Minor perf improvements.
- Cache the vector sizes in IExecutionFrame and NodeIndexInfo to avoid calls to size().
- 2 instructions instead of 10
- Remove an unnecessary check in IExecutionFrame
- add a check to the ctor so we guarantee it's unnecessary
- Reserve memory for the vectors in BroadcastIterator
- saves reallocs if more than one value is added
- but rare with the mlperf models for multiple values to be added so benefit is limited.
- slight tweak to the Broadcaster ctor code to make it more readable
* Mention OrtCreateSessionFromArray in C API doc
* Fix perf test executable due to removal of certain C APIs
* fix linux build
* Avoid duplication
* Fix mem leak
* Update nGraph to 0.21 and adjust the EP
* Share the graph initializers between custom ops
* Update nGraph to 0.22 and exclude Gather entirely
* Enable building on Windows with nGraph v0.21.1-rc.0
* Disable the unsigned input Shrink op tests for nGraph until the next update
* Line-shortening code refactor
* Fix for the master branch merge artifact
* MKLDNN patches adjustment for Windows
* Exclude MatMulInteger for non-const zero points
* Exclude ConvInteger for non-const zero points
* Enable full Cast op support
* Use the v0.22.1 tag
* Skip ConvTranspose_InvalidKernelShape test for ngraph provider
* Create sub-graph ModelProto from fused_node
* Implement new LabelEncoder in opset 2 in ML domain
* Fix compilation error
* Fix tests
* Include ONNX's fix
* Formatting and addressing a comment
* Address a minor comment
* For majority of nodes, we do not need to do fence check. Instead, we only need to do FenceCheck for CPU<->GPU mem sync node
But we pay the Fence check cost for every single node and every single input and output.
This change will minimize the Fence check to only do it when necessary.
* remove memory copy between CUDA and TRT
* add info to RegisterExecutionProvider input
* use new IDeviceAllocator for trt allocator
* remove SetDefaultInputsMemoryType from TRT EP
* remove onnx-tensorrt 5.0
* add submodule onnx-tensorrt branch 5.1
* remove redundancy
* Update transformer_memcpy.cc
* Update tensorrt_execution_provider.cc
* switch to TensorRT 5.1.5.0
* update python binding
* disable failed test case on TensorRT
* Update activation_op_test.cc
* upgrade to TensorRT container 19.06
* update according to feedback
* add comments
* remove tensorrt allocator and use cuda(gpu) allocator
* update onnx-tensorrt submodule
* change ci build cuda directory name
Fix race condition issue in RNN/LSTM/GRU.
Description:
The filter_desc and rnn_desc could also be changed in compute which could be in multi-thread. It will cause race condition issue.
Fix:
create temperate cudnn descriptors
cache cudnn_dropout_desc_ which won't change
* A few performance improvements:
- Make the iteration in NonZero more efficient by using a raw pointer and simplifying the increment logic
- add another unit test to check the new logic works with 3 dimensional tensor
- gains about 2% for ssd_mobilenet
- Avoid floating point operations on each iteration on Concat
- about 0.5% for ssd_mobilenet and ssd_resnet34
- Put common case first in ExecutionFrame::AllocateAsPerAllocationPlan to avoid unnecessary call to IsSparseTensor
- about 0.05% for ssd_mobilenet
- Minor tweak to put some ctors in the TensorShape header so they can be inlined more easily
* If there is an outer scope value that matches a subgraph input, don't create an implicit input from the outer scope value.
Minor unrelated change for issue noticed while debugging: Use unordered_set for implicit inputs so we don't add them multiple times.
* Add unit test based on onnx issue.
* Bug fix for shape of optional output in Dropout op
* Exclude new test from NGraph EP
* Account for the fact that mask could be of different type in different opset variants of the op
* Make accompanying Cuda changes
* Fix build break
* Exclude Opset 7 test for tensorRT EP
* PR comments
Description:
crash if the output shape has 0 in it. because the code to / output_shape[i]
Fix:
If the output shape has 0 which means output_shape.Size() is 0, so output should be null.
* Add MacOS leg of Python packaging job
* Update copy files source directory for Mac OS leg
* Add a task to display the binaries directories contents after build wheel creation
* Revert some changes
* Add task to log
* Update
* Remove unnecessary logs
Python script and necessary changes in the azure-pipelines yaml file to post the binary size data from NuGet package build. Currently only posted from CPU pipeline. GPU and other pipelines may be added as necessary.
1. Move non_max_suppression_test.cc to object_detection folder
2. Move Class CudnnDropout to cudnn_common.h so that can share it with other ops. Move the cuda memory allocation part out of CudnnDropout to avoid memory leak.
