onnxruntime/onnxruntime/test/onnx
Weixing Zhang 299ace0759
Support to allow user to specify compute stream per session (#3723)
* Support to allow user to specify compute stream per session

Create computation cuda stream explicitly rather than use default legacy stream or per-thread default stream.

remove some redudant cudaStreamSynchronize

fix gpt2 model test failures

don't use default stream in nccl either.

add stream schronization in OnRunEnd()

using cub::DeviceScan::InclusiveSum which can be called with stream specified.

fix topK failure due to latest rebase

fix tensorrt

support user specified stream

add user_stream support in tensorrt EP

use same stream for both tensort and CUDA EP.

fix ScatterND

specify stream for adasum and p2p kernels.

fix loop

fix CApiTest.custom_op_handler

fix CApiTest.varied_input_custom_op_handler

change for cudaMemcpyFromSymbol

improve provider options for user specified compute stream

* add changes for ROCM EP

* fix GatherGrad UT for ROCM EP

* clean code and fix NonMaxSuppression

* use default stream for ROCM now

* fix CApiTest.custom_op_handler:OrtFormatCustomOpTests.ConvertOnnxModelToOrt

* fix tensorrt ut: CApiTest.io_binding_cuda

Co-authored-by: Weixing Zhang <wezhan@microsoft.com>
2021-02-05 15:48:18 -08:00
..
microbenchmark Avoid false sharing on thread pool data structures (#6298) 2021-01-12 19:58:41 +00:00
callables.h Refactor onnx_test_runner (#5169) 2020-09-18 13:19:35 -07:00
callback.cc Refactor onnx_test_runner (#5169) 2020-09-18 13:19:35 -07:00
callback.h Refactor onnx_test_runner (#5169) 2020-09-18 13:19:35 -07:00
dataitem_request.cc ThreadPool clean up : mm_pause in loops, correctly spin-then-wait, and adopt static methods consistently in the API (#5590) 2020-10-28 09:49:18 +00:00
dataitem_request.h Refactor onnx_test_runner (#5169) 2020-09-18 13:19:35 -07:00
gen_test_models.py Remove openmp and scipy from build pipelines (#4305) 2020-06-23 20:18:16 -07:00
heap_buffer.cc Refactor onnx_test_runner (#5169) 2020-09-18 13:19:35 -07:00
heap_buffer.h Refactor onnx_test_runner (#5169) 2020-09-18 13:19:35 -07:00
main.cc Support to allow user to specify compute stream per session (#3723) 2021-02-05 15:48:18 -08:00
mem_buffer.h Refactor onnx_test_runner (#5169) 2020-09-18 13:19:35 -07:00
onnx_model_info.cc Support opset-13 specs of controlflow ops (Loop, If) (#5665) 2020-11-11 23:44:14 -08:00
onnx_model_info.h Support opset-13 specs of controlflow ops (Loop, If) (#5665) 2020-11-11 23:44:14 -08:00
onnxruntime_event.h
OrtValueList.h
pb_helper.cc
pb_helper.h Support opset-13 specs of controlflow ops (Loop, If) (#5665) 2020-11-11 23:44:14 -08:00
README.txt Remove nGraph Execution Provider (#5858) 2020-11-19 16:47:55 -08:00
tensorprotoutils.cc Deprecate Python global configuration functions [Part 2] (#6171) 2021-01-07 10:10:55 -08:00
tensorprotoutils.h Refactor onnx_test_runner (#5169) 2020-09-18 13:19:35 -07:00
TestCase.cc Support opset-13 specs of controlflow ops (Loop, If) (#5665) 2020-11-11 23:44:14 -08:00
TestCase.h Support opset-13 specs of controlflow ops (Loop, If) (#5665) 2020-11-11 23:44:14 -08:00
testcase_driver.cc Preserve relative order of the results and the tests. (#5225) 2020-09-19 00:45:44 -07:00
testcase_driver.h Preserve relative order of the results and the tests. (#5225) 2020-09-19 00:45:44 -07:00
testcase_request.cc ThreadPool clean up : mm_pause in loops, correctly spin-then-wait, and adopt static methods consistently in the API (#5590) 2020-10-28 09:49:18 +00:00
testcase_request.h Preserve relative order of the results and the tests. (#5225) 2020-09-19 00:45:44 -07:00
TestCaseResult.cc Refactor onnx_test_runner (#5169) 2020-09-18 13:19:35 -07:00
TestCaseResult.h Refactor onnx_test_runner (#5169) 2020-09-18 13:19:35 -07:00
testenv.cc Convert OpenVINO into a shared provider (#5778) 2020-11-20 17:39:57 -08:00
testenv.h Refactor onnx_test_runner (#5169) 2020-09-18 13:19:35 -07:00
TestResultStat.cc
TestResultStat.h

onnx_test_runner [options...] <data_root>
Options:
        -j [models]: Specifies the number of models to run simultaneously.
        -c [runs]: Specifies the number of Session::Run() to invoke simultaneously for each model.
        -n [test_case_name]: Specifies a single test case to run.
        -p [PLANNER_TYPE]: PLANNER_TYPE could be 'seq' or 'simple'. Default: 'simple'.
        -e [EXECUTION_PROVIDER]: EXECUTION_PROVIDER could be 'cpu', 'cuda', 'dnnl', 'tensorrt', 'nuphar' or 'acl'. Default: 'cpu'.
        -h: help

The debug version of this program depends on dbghelp.dll. Please make sure it's in your PATH.

How to run node tests:
1. Install onnx from onnxruntime\cmake\external\onnx

2. Execute test data generator:
       backend-test-tools generate-data -o <some_empty_folder>
   e.g.
       backend-test-tools generate-data -o C:\testdata
    backend-test-tools is a tool under C:\Python35\Scripts (If your python was installed to C:\Python35)

3. compile onnx_test_runner and run
      onnx_test_runner <test_data_dir>
	e.g.
	  onnx_test_runner C:\testdata\node


How to run model tests:
1. Download the test data from Azure
   You can get the latest url from tools/ci_build/github/azure-pipelines/templates/set-test-data-variables-step.yml
   After downloading, please unzip the downloaded file

2. compile onnx_test_runner and run
   onnx_test_runner <test_data_dir>
   e.g.
	 onnx_test_runner C:\testdata