Commit graph

3693 commits

Author SHA1 Message Date
Ryan Lai
697e8faa9e
Skip failing x86 winml tests and update testData environment variable path mechanism (#5719)
* Skip failing x86 winml tests

* fix gpt2 rename typo

* there are actually 2 gpt model tests
2020-11-06 13:59:29 -08:00
Johannes Bannhofer
9ec6da1e27
added missing flag ORT_TENSORRT_DUMP_SUBGRAPHS (#5724)
[DOCUMENTATION]
added descriptionof the function ORT_TENSORRT_DUMP_SUBGRAPHS to the documentation
2020-11-06 12:33:18 -08:00
Johannes Bannhofer
6f6dd0b869
added missing flag ORT_TENSORRT_DUMP_SUBGRAPHS (#5724)
[DOCUMENTATION]
added descriptionof the function ORT_TENSORRT_DUMP_SUBGRAPHS to the documentation
2020-11-06 12:32:12 -08:00
Chi Lo
92292de135
Tensorrt perf tool (#5436)
* Add YAML file for pipeline

* Modify typo

* Add working directory

* Modify and test

* Modfiy and test

* Modify and test

* Modify and test

* Modify

* Modify

* Modify

* Modify

* Make sure to copy all the result files

* Add clearn up

* Modify

* Modify agent pool name

* Upload only specific artifacts

* Modify

* Integrated CI Pipeline for running TRT perf as well as added the “large amount of models” into perf model target

* Fix bug

* Fix bug

* Add reading the information regarding previously known failing models
and then skip testing them during benchmark/validation

* Modify the script file for CI

* Replace print with logger.info

* Fix bug

* Fix bug

* Refine the code

* Modify the script so that it can capture script segmentation fault while
running ORT

* Fix bug

* fix bug

* fix bug

* Add debug info

* fix bug

* Refine perf code

* Refine the code

* fix bug

* Code refactoring

* change many-models path

* remove metadata after validation/benchmark are done

* Update README.md

* Fix bug so that metadata doesn't hold stale value

* Remove hardcode and update README

* Add arguments to the script to make it run correctly

* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines

* Update linux-gpu-tensorrt-ci-perf-pipeline.yml for Azure Pipelines

* Fix bug so that metadata doesn't hold stale value

* Fix small bug of finding test dataset directory for FP16 test data, as
well as modification of some output information

* use -i random for perf test of TRT changes

Co-authored-by: Olivia Jain <oljain@microsoft.com>
2020-11-06 12:27:42 -08:00
Ye Wang
95e6da7957
Revert saving optimized model as external data (#5690)
* revert and add support for saving external data

* review comments

* update
2020-11-06 11:54:19 -08:00
RandySheriffH
71f90e08f1
Nuget packaging no omp (#5666)
* create new nuget packaging pipeline without openmp

* rename package

* update image name

* rename package name

* rename managed package

* reset project attribute

* merge master

* set package name

* set NoOpenMP as cpu build

* shorten line length

Co-authored-by: Randy Shuai <rashuai@microsoft.com>
2020-11-06 11:43:35 -08:00
Zhang Lei
77b1eea9cf
Add option to allow quantize_input() use input_qtype for initializers. (#5721) 2020-11-06 09:33:24 -08:00
George Wu
f666c3d7d7
update jetson build instructions (#5725) 2020-11-06 09:33:04 -08:00
Zhang Lei
24016a517b
Prepacking in Gemm with merged logic for Matmul and Gemm on PackingB. (#5693)
Prepacking in Gemm with merged logic for Matmul and Gemm on PackingB.
2020-11-05 22:35:24 -08:00
Nat Kershaw (MSFT)
479ed740ef
Add link to survey to README (#5685)
* Add survey request to README

* Remove period

* Fix #5681 - broken link
2020-11-05 18:01:08 -08:00
Maajid khan
d6f9cc181d
Modify logic to determine OV Version (#5701)
Signed-off-by: MaajidKhan <n.maajidkhan@gmail.com>
2020-11-05 15:12:02 -08:00
Adam Pocock
d1d82065b9
[Java] Fixes an error allocating large direct byte buffers during OnnxTensor creation (#5619)
* Fixing an error with allocating large direct byte buffers during tensor creation.

* Removing the redundant overflow check.
2020-11-05 15:02:41 -08:00
Pranav Sharma
28197b1460
Register opset13 flatten, LRN for cuda. (#5694)
* Register opset13 flatten, LRN, ArgMax and ArgMin for cuda.

* Fix build
2020-11-05 14:13:15 -08:00
Scott McKay
11fe683471
Partition full graph one execution provider at a time (#5635)
* Partition full graph one EP at a time, bottom-up. Nuphar requires this and it makes life simpler for an EP as they can just check if all nodes in a subgraph are assigned to it when processing the control flow node containing the subgraph.

Make a couple of nuphar error messages more meaningful.
2020-11-06 07:26:00 +10:00
edgchen1
858040faaa
Implement reduce_matrix_columns() to optimize ReduceSum (#5639)
Implement reduce_matrix_columns() to optimize ReduceSum.
2020-11-05 10:25:00 -08:00
George Wu
c46515cd56
[TensorRT EP] Remove cudaDeviceSynchronize and use cudaAllocator for scratch buffers (#5714)
* use cuda allocator, remove cudaDeviceSync call

* use unique_ptr for scratch buffers
2020-11-05 09:45:27 -08:00
Dmitri Smirnov
fd9d0c4ee0
Remove redundant const_cast (#5705)
Signed-off-by: Dmitri Smirnov <dmitrism@microsoft.com>
2020-11-05 09:43:22 -08:00
Tiago Koji Castro Shibata
9e68e98423
Add static CRT DLLs to Nuget package (#5661)
* Add static runtime yaml option

* Add to WAI Nuget build matrix

* Support empty build flags

* Add DML to x64

* Bundle static rt

* Bundle after Nugets are built

* Fix typo

* Skip static tests

* Pack test artifact only in x64 dynamic

* No DML static runtime

* Add Store static

* Revert "Add Store static"

This reverts commit 69133e5838.

* Static subfolder
2020-11-05 09:26:17 -08:00
Tim Harris
ff23083de2
Unbreak microbenchmark build (#5710)
Minor updates to the microbenchmarks built optionally with "--build_micro_benchmarks". These are not built as part of CI, and builds started to fail. There are three changes:

- I updated the threading-related benchmarks to use the static-method ThreadPool API, and to expose control over the thread pool configuration via constexpr int variables.

- Disable GCC warnings seen with recent compiler versions when including parts of the Eigen headers in batchnorm.cc and eigen.cc files.

- Flush std::cerr on error conditions to avoid buffered messages being lost.

I tested manual builds with Linux (GCC) and Windows (MSVC).
2020-11-05 10:46:59 +00:00
Yufeng Li
5c4543e194
Calibrate float tensor only (#5704) 2020-11-04 23:55:48 -08:00
Scott McKay
2127a229d7
The IndexedSubGraph is used to create the Function body, but after that is invalid as the nodes it referred to have been removed from the main Graph. As such there's no need to store it in the FunctionImpl instance. (#5669) 2020-11-05 17:21:56 +10:00
Ryan Hill
941e3a69f9
Test a build break fix (#5706) 2020-11-04 21:15:38 -08:00
ashbhandare
6d8e81cb08
Update Squeeze, Unsqueeze, Split and ReduceSum kernel for Opset13 (#5691)
* Split  change

* ReduceSum and Split change

* Other op changes, Grad builder, tests, registering required opset 13 ops

* Rebase fixes

* Fix tests, add some more

* Review changes, rebase

* Fix windows build

* Disable new tests for TesnorRT EP

* Disable unsupported for OpenVINO

Co-authored-by: Aishwarya <aibhanda@OrtTrainingDev4.af05slrtruoetgaxwwjv5nsq5e.px.internal.cloudapp.net>
2020-11-04 20:00:27 -08:00
Dmitri Smirnov
830f567be8
Add C API Guidelines document (#5686)
Add C API Guidelines document
Signed-off-by: Dmitri Smirnov <dmitrism@microsoft.com>
2020-11-04 18:50:31 -08:00
alexzakv
8bae883d3e
User/alexzak/win ml principles (#5453)
* Contributing page change

* Update WinML_principles.md

* Update WinML_principles.md

* Update WinML_principles.md

* Updated

* Update WinML_principles.md

* Update WinML_principles.md

* Update WinML_principles.md
2020-11-04 13:35:40 -08:00
wezuo
62a99824cb
Wezuo/priority in nodedef (#5692)
* set the priority in nodedef

* remove debugging stmts

* revoke zero builder

* remove unnecessary namespace comment

Co-authored-by: wezuo <wezuo@az-eus-v100-32gb-5-worker-mgtbby.eastus.cloudapp.azure.com>
Co-authored-by: Wei Zuo <wezuo@OrtTrainingDev3.af05slrtruoetgaxwwjv5nsq5e.px.internal.cloudapp.net>
2020-11-04 12:40:37 -08:00
S. Manohar Karlapalem
e49f7a8b71
Fix build error due to unused variable (#5698)
Fixes build error due to unused variable when building with
OpenVINO 2020.2 and 2020.3.
2020-11-04 12:12:16 -08:00
Changming Sun
0b9f7bb1b0 Update InferenceTest.cs 2020-11-04 11:39:49 -08:00
Changming Sun
0445473dc1 Add ssd to x86_disabled_tests 2020-11-04 11:39:49 -08:00
Guoyu Wang
a2b551ff08
Add runtime options for NNAPI EP (#5576)
* Add options for nnapi ep

* Add nnapi flags test

* add comments

* Add flag comments

* Make the flags bitset const

* Fix build break

* Add stub changes to java and c# api

* Fix java related build break

* Fix java build break

* Switch to bit flags instead of bitset
2020-11-04 10:08:43 -08:00
Guoyu Wang
2ad7bcb766
NNAPI add opset version check (#5687)
* nnapi add opset support
2020-11-04 21:48:00 +10:00
edgchen1
07bd4ef470
Upgrade optional implementation to https://github.com/martinmoene/optional-lite. (#5563) 2020-11-03 15:27:47 -08:00
Changming Sun
67d7e3967d Disable some model tests 2020-11-03 14:42:45 -08:00
Hector Li
b6eeadf420
Enable OpenVino build on Arm64 platform (#5682) 2020-11-03 13:55:34 -08:00
Scott McKay
c9f44276da
Add ability to filter GraphViewer using IndexedSubGraph. (#5614)
* Add ability to filter GraphViewer using IndexedSubGraph. This is to support compiling execution providers in a minimal build.
2020-11-04 07:08:18 +10:00
Changming Sun
357a51c75c
Update python packaging pipeline's docker image (#5680) 2020-11-03 12:01:36 -08:00
Hariharan Seshadri
db9c1308a5
Fix Resize kernel registration (#5677) 2020-11-03 10:43:41 -08:00
edgchen1
28f1e32898
Loosen tolerance of CudaKernelTest.ReduceSum_MidTensor, allow test random seed to be regenerated within a test run. (#5675) 2020-11-03 10:37:00 -08:00
Ye Wang
a028ca41ec
Optimize flaubert (#5651)
* optimize flaubert

* fix an issue and format

* revert non-relevent change

* review comments
2020-11-03 09:51:42 -08:00
M. Zeeshan Siddiqui
9b010963b7
Turn off peak memory logging and fix memory pattern generation bug. (#5676)
* Turn off peak memory log lines and fix memory pattern generation bug.

* Turn off peak memory log lines and fix memory pattern generation bug.

Co-authored-by: Ubuntu <OrtTrainingDev3@OrtTrainingDev3.af05slrtruoetgaxwwjv5nsq5e.px.internal.cloudapp.net>
2020-11-03 08:44:15 -08:00
Dmitri Smirnov
5d66cf017c
Register Clip for OpSet13 (#5671)
Signed-off-by: Dmitri Smirnov <dmitrism@microsoft.com>
2020-11-03 07:07:28 -08:00
Wei-Sheng Chin
8856c2595b
Sync the two IDs in OrtMemoryInfo when calling ctor (#5663)
* Sync the two IDs in OrtMemoryInfo when calling ctor

* Also fix the same problem for output
2020-11-02 23:22:47 -08:00
Changming Sun
4936e10e22
Disable some model tests (#5664)
These are the new models added by WinML team. But some of our EPs can't pass some of tests.
2020-11-02 22:01:35 -08:00
Tracy Sharpe
182d9c48e4
Merge u8u8/u8s8 QLinearConv implementations (#5662)
Combine the u8u8/u8s8 implementations for x86/x64 builds and add special case handling for 1D convolutions.
2020-11-02 21:38:39 -08:00
ashbhandare
c875fe0919
Add option to dump activations on all ranks (#5455)
* Add option to dump activations on all ranks

* address review comments

* review comments

* Fix review comment

Co-authored-by: Aishwarya <aibhanda@OrtTrainingDev4.af05slrtruoetgaxwwjv5nsq5e.px.internal.cloudapp.net>
2020-11-02 18:03:05 -08:00
Changming Sun
87e1063e19
Revert "Update Squeeze, Unsqueeze, Split and ReduceSum kernel for Opset13 (#5488)" (#5668)
This reverts commit db63c5d10f.
2020-11-02 16:09:22 -08:00
Tianlei Wu
2c02530603
Bert Model Profiling Tool (#5654)
* Add profiler tool for BERT models
2020-11-02 13:47:37 -08:00
Jesse Benson
1495f737ca Use cudaMemsetAsync and add checks on CUDA calls. 2020-11-02 11:25:13 -08:00
ashbhandare
db63c5d10f
Update Squeeze, Unsqueeze, Split and ReduceSum kernel for Opset13 (#5488)
* Split  change

* ReduceSum and Split change

* Other op changes, Grad builder, tests, registering required opset 13 ops

* Rebase fixes

* Fix tests, add some more

* Review changes, rebase

* Fix windows build

Co-authored-by: Aishwarya <aibhanda@OrtTrainingDev4.af05slrtruoetgaxwwjv5nsq5e.px.internal.cloudapp.net>
2020-11-02 10:51:48 -08:00
Wenbing Li
5b44982971
Change the OrtCustomOp invocation as a constant. (#5506)
* Chanage the OrtCustomOp invocation as a constant.

* fix build on macos

* build fixing
2020-11-02 10:38:07 -08:00