* checkin
* fix MSVC build error
* test changes
* split pivot output into multiple tensors
* add horizon tensor
* Support multiple types for non-pivot tensor
* limit horizon tensor type to int32_t as max_horizon type
* work around some conversion warnings for local machine
* support variadic shape for non-pivot input
* dropping all rows is an exception
* fix a bug
* fix the way that generates horizon tensor
* more tests added
* add TypeConstraint() in ONNX_OPERATOR_KERNEL_EX
* update Featurizerslibrary
* Remove paramters like --gpu_only --sequence_length. Update bert GPU notebook accordingly.
* Remove input_int32 and float16 parameters from constructors of BertOnnxModel class and other classes derived from it.
* Update gpt2 benchmark. Add comments in gpt2 notebook to indicate work in progress. Clear notebook output before official 1.3.0 release is ready.
* Update TopK implementation.
- add faster heap
- special case k=1
- update selector for when to use heap and when to use nth_element based on performance testing
- parallelize if enough work to do
- reduce templatized code
- add some extra unit tests.
Perf tested vs. master. Average speedup is 3.75x using this combination of input sizes:
```
batches = [10, 25, 50]
batch_size = [8, 16, 32, 64, 128, 256, 512, 1024, 2048]
k = [1, 2, 4, 6, 8, 16, 24, 32, 48, 64, 128]
```
For larger batches (e.g. 50x2048) the speedup is over 20x.
Threadpool related changes.
Don't create ORT threadpool if openmp is enabled (except for inter op threadpool).
Created a new static function ThreadPool::NumThreads to account for openmp settings and null threadpool ptr.
Log a warning when using SetIntraOpNumThreads when openmp is enabled.
Added a document for ORT devs.
Fix LSTM to use the new threadpool abstractions.
Rename GetNumCpuCores to GetThreadAffinityMasks and move it to the Env class.
Co-authored-by: Tracy Sharpe <tracysh@microsoft.com>
* Generalize reshape fusion
* Allow arbitrary number of Concat arguments
* Apply fusion even when an output of an internal node is used elsewhere
* Fix a bug when an internal node's output is the subgraph output
* Simplify code
* Add Attention fusion for GPT2
* Support distilgpt2 in benchmark_gpt2.py
* Add options to disable Attention/SkipLayerNormalization/EmbedLayerNormalization/BiasGelu fusions
* Add logging at the begining of each fusion
* Update notebooks: Add Gpt2OnnxModel.py to list of script files.
* Add test for gpt2 model optimization
* Add optional parameters (--input_ids --segment_ids --input_mask) for graph inputs
* Fuse BiasGelu
* Handle model that does not have segment_ids input.
* Allow fuse embed layer without mask
* Make QuantizeLinear support half
* remove unnessary type constraint
* refine kernel definition
* add fp16 support for dequantizelinear
* diable QuantizeLinear_per_tensor_half_int8 for tensorrt
* refine unit test and fix saturate issue for MSDomain QuantizeLinear
* fix build break
* include tensorrt for half_uint8 test
* Migrate winml to Microsoft Namespace (packaging changes are pending)
* add ns_prefix toggle
* fix packaging
* Users/sheilk/add missing raw header (#3484)
* add dualapipartition
* wrong variable for repo root
Co-authored-by: Sheil Kumar <sheilk@microsoft.com>
* remove existence check to force failures
* extra paren
* dualapipartition needs to be referenced from the source
* add microsoft.ai.machinelearning.dll to the output dir
* rename the idl file so that assembly info is correctly added into the winmd
* fix namespaces
* update namespaces
* default to microsoft, and add namespace override as build argument
* update cmakesetings.json as well
* remove from cmakelists.txt
Co-authored-by: Sheil Kumar <sheilk@microsoft.com>
Co-authored-by: Changming Sun <chasun@microsoft.com>
* Fixed cornercases for acl ep gemm implementation by setting fully connected as the main layer
* Introduced versioned build for the acl ep. ACL versions supported are 1902, 1905 and 1908
* Added convolution-activation fusion optimization for acl ep. We see improvements of 12% for mobilenetv2 and 4% for resnet50
Co-authored-by: Andrei-Alexandru <andrei-alexandru.avram@nxp.com>
1. Fix static analysis warnings found by VC++
2. Add a new pipeline for static analysis
3. Merge all the windows CI build into one single yaml file.(Easier to queue them all).
4. Make DNNL build faster by disabling building the tests and examples.
5. Enable custom op unitest.
