Commit graph

14 commits

Author SHA1 Message Date
Yi Zhang
a9a9c34d98
Fix WinML Test Case: create LearningModelBinding for every testcase (#13587)
### Description
Fix #13509

### Motivation and Context
The exception was caused by the incorrect fetches, which was from the
binding with last test cases.

efcbdac58e/onnxruntime/core/session/onnxruntime_c_api.cc (L809-L815)
2022-11-09 11:20:48 +08:00
Yi Zhang
e160688a9b
Skip some failed models winml and training workflows on Windows CPU (#13407)
### Description
1. update model name structure in model_tests.cpp with source name. To
avoid
`Condition test_param_names.count(param_name) == 0 failed. Duplicate
parameterized test name 'BERT_Squad_opset10_CPU'`
2. skip some failed models https://github.com/onnx/models/issues/568


### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
2022-10-25 10:05:04 +08:00
Chun-Wei Chen
6246662b1d
[Dup] Fix SAME_UPPER/SAME_LOWER (auto_pad attribute) in ConvTranspose (#12537)
* Fix SAME_UPPER/SAME_LOWER (auto_pad attribute) in ConvTranspose

* Bump ONNX 1.10.2 globally

* load ONNX_VERSION from VERSION_NUMBER

* /

* revert deprecate warning in ORT 1.12

* add a comment about why removing cntk_simple_seg

* correct the implem in DML as well
2022-08-22 15:35:34 -07:00
Gary Miguel
e8b0d24071
Support per-test tolerances for ONNX tests (#11775)
Prior to this every test shared the same tolerances. This meant
that if an ONNX test failed due to a small but acceptable difference in
output, the only alternative was to disable the test entirely.

In op set 17, the DFT operator is being added. Without this change, the
tests for that operator fail because the output is off by about 5e-5.
It's better to keep test coverage for this new op rather than disable
the test entirely.

Also prior to this change, the global tolerances were not shared between
C++, JavaScript, and Python tests. Now they are.

Also fix various minor issues raised by linters.

Unblocks https://github.com/microsoft/onnxruntime/issues/11640.
2022-06-14 15:12:23 -07:00
Dwayne Robinson
7a3abd863f Update WinML model test tolerances for tiny_yolov2 and FNS_Candy 2021-12-02 00:48:54 -08:00
sumitsays
e344a583b0
updated sampleTolerance of model fp16_inception_v1 for GPU execution provider (#7533)
Co-authored-by: Sumit Agarwal <sumitagarwal@microsoft.com>
2021-05-03 12:08:31 -07:00
Ryan Lai
10102c09b6
Add better model test error messaging (#7239) 2021-04-05 14:59:19 -07:00
Ryan Lai
0494a0f95f
Add ability to skip GPU tests based on GPU adapter name (#6198)
* Implement conversion from ortvalue to Itensor for string tensors and comparing sequence of maps of strings to floats

* PR comments

* Add ability to skip gpu tests according to adapter description

* spacing

* spacing

* spacing
2020-12-22 15:20:23 -08:00
Ryan Lai
21395f8e24
Implement comparing outputs that are sequence of maps of strings to floats (#6180)
* Implement conversion from ortvalue to Itensor for string tensors and comparing sequence of maps of strings to floats

* PR comments
2020-12-22 09:52:29 -08:00
Ryan Lai
753af576c4
If building inbox, hook up winrt_activation_handler for WinML Tests (#6074)
* If building inbox, hook up winrt_activation_handler with what is already defined in in dllload.cpp

* Add base.h header

* Missed custom ops test
2020-12-10 14:41:01 -08:00
Ryan Lai
e40df385ba
Skipping even more x86 tests (#5799) 2020-11-15 20:52:26 -08:00
Ryan Lai
697e8faa9e
Skip failing x86 winml tests and update testData environment variable path mechanism (#5719)
* Skip failing x86 winml tests

* fix gpt2 rename typo

* there are actually 2 gpt model tests
2020-11-06 13:59:29 -08:00
Ryan Lai
98538580c8
give more tolerance to DirectML runs (#5564) 2020-10-21 23:14:51 -07:00
Ryan Lai
f207f0bf5e
Add WinML Model testing (#5417)
* Model test start with float

* Clean up code and add environment variable detection

* Move into namespace

* PR comments

* Fix linker errors in latest merge to master and also fix warning

* add skipping model test mechanism

* Return std::string instead of writing to buffer

* Address case where env variable is larger than max_path

* use const static string for test reason

* Disable x86 tests and don't build if ort memory checker is enabled

* Add comment

* Add additional failing x86 tests and ifdef for checking fo rx86 build

* PR comments
2020-10-15 19:04:12 -07:00