saymrwulf/onnxruntime
1
0
Fork 0
mirror of https://github.com/saymrwulf/onnxruntime.git synced 2026-05-16 21:00:14 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
onnxruntime/docs/How_To_Update_ONNX_Dev_Notes.md

45 lines
2.6 KiB
Markdown
Raw Normal View History

Support per-test tolerances for ONNX tests (#11775) Prior to this every test shared the same tolerances. This meant that if an ONNX test failed due to a small but acceptable difference in output, the only alternative was to disable the test entirely. In op set 17, the DFT operator is being added. Without this change, the tests for that operator fail because the output is off by about 5e-5. It's better to keep test coverage for this new op rather than disable the test entirely. Also prior to this change, the global tolerances were not shared between C++, JavaScript, and Python tests. Now they are. Also fix various minor issues raised by linters. Unblocks https://github.com/microsoft/onnxruntime/issues/11640.
2022-06-14 22:12:23 +00:00
# How to update ONNX
Add dev notes
2019-02-01 22:16:30 +00:00
Support per-test tolerances for ONNX tests (#11775) Prior to this every test shared the same tolerances. This meant that if an ONNX test failed due to a small but acceptable difference in output, the only alternative was to disable the test entirely. In op set 17, the DFT operator is being added. Without this change, the tests for that operator fail because the output is off by about 5e-5. It's better to keep test coverage for this new op rather than disable the test entirely. Also prior to this change, the global tolerances were not shared between C++, JavaScript, and Python tests. Now they are. Also fix various minor issues raised by linters. Unblocks https://github.com/microsoft/onnxruntime/issues/11640.
2022-06-14 22:12:23 +00:00
This note is only for ONNX Runtime developers.
Add dev notes
2019-02-01 22:16:30 +00:00
Support per-test tolerances for ONNX tests (#11775) Prior to this every test shared the same tolerances. This meant that if an ONNX test failed due to a small but acceptable difference in output, the only alternative was to disable the test entirely. In op set 17, the DFT operator is being added. Without this change, the tests for that operator fail because the output is off by about 5e-5. It's better to keep test coverage for this new op rather than disable the test entirely. Also prior to this change, the global tolerances were not shared between C++, JavaScript, and Python tests. Now they are. Also fix various minor issues raised by linters. Unblocks https://github.com/microsoft/onnxruntime/issues/11640.
2022-06-14 22:12:23 +00:00
If you need to update the ONNX submodule to a different version, follow the steps below.
1. Update the ONNX submodule
```sh
Add dev notes
2019-02-01 22:16:30 +00:00
cd cmake/external/onnx
git remote update
git reset --hard <commit_id>
cd ..
git add onnx
```
(Change the <commit_id> to yours. If you are not sure, use 'origin/master'. Like 'git reset --hard origin/master')
Fix C# and numbering (#11643) * C# protocol buffer code can be updated on Linux. Link to the relevant instructions. * Fix numbering.
2022-05-31 18:33:36 +00:00
1. Update [cgmanifests/generated/cgmanifest.json](/cgmanifests/generated/cgmanifest.json).
CGManifest - add training entries and generate entries for submodules. (#3933) Add cgmanifest.json entries for training dependencies. Add script to generate git submodule cgmanifest.json entries.
2020-05-15 20:34:18 +00:00
This file should be generated. See [cgmanifests/README](/cgmanifests/README.md) for instructions.
Add dev notes
2019-02-01 22:16:30 +00:00
Support per-test tolerances for ONNX tests (#11775) Prior to this every test shared the same tolerances. This meant that if an ONNX test failed due to a small but acceptable difference in output, the only alternative was to disable the test entirely. In op set 17, the DFT operator is being added. Without this change, the tests for that operator fail because the output is off by about 5e-5. It's better to keep test coverage for this new op rather than disable the test entirely. Also prior to this change, the global tolerances were not shared between C++, JavaScript, and Python tests. Now they are. Also fix various minor issues raised by linters. Unblocks https://github.com/microsoft/onnxruntime/issues/11640.
2022-06-14 22:12:23 +00:00
1. Update [tools/ci_build/github/linux/docker/scripts/requirements.txt](/tools/ci_build/github/linux/docker/scripts/requirements.txt)
and [tools/ci_build/github/linux/docker/scripts/manylinux/requirements.txt](/tools/ci_build/github/linux/docker/scripts/manylinux/requirements.txt).
Update the commit hash for `git+http://github.com/onnx/onnx.git@targetonnxcommithash#egg=onnx`.
Add dev notes
2019-02-01 22:16:30 +00:00
Support per-test tolerances for ONNX tests (#11775) Prior to this every test shared the same tolerances. This meant that if an ONNX test failed due to a small but acceptable difference in output, the only alternative was to disable the test entirely. In op set 17, the DFT operator is being added. Without this change, the tests for that operator fail because the output is off by about 5e-5. It's better to keep test coverage for this new op rather than disable the test entirely. Also prior to this change, the global tolerances were not shared between C++, JavaScript, and Python tests. Now they are. Also fix various minor issues raised by linters. Unblocks https://github.com/microsoft/onnxruntime/issues/11640.
2022-06-14 22:12:23 +00:00
1. If there is any change to `cmake/external/onnx/onnx/*.in.proto`, you need to regenerate OnnxMl.cs.
[Building onnxruntime with Nuget](https://onnxruntime.ai/docs/build/inferencing.html#build-nuget-packages) will do
this.
Remove onnxruntime/core/protobuf (#8617) * remove onnxruntime/core/protobuf * Update How_To_Update_ONNX_Dev_Notes.md
2021-08-10 16:36:27 +00:00
Support per-test tolerances for ONNX tests (#11775) Prior to this every test shared the same tolerances. This meant that if an ONNX test failed due to a small but acceptable difference in output, the only alternative was to disable the test entirely. In op set 17, the DFT operator is being added. Without this change, the tests for that operator fail because the output is off by about 5e-5. It's better to keep test coverage for this new op rather than disable the test entirely. Also prior to this change, the global tolerances were not shared between C++, JavaScript, and Python tests. Now they are. Also fix various minor issues raised by linters. Unblocks https://github.com/microsoft/onnxruntime/issues/11640.
2022-06-14 22:12:23 +00:00
1. If you are updating ONNX from a released tag to a new commit, please ask Changming (@snnn) to deploy the new test
data along with other test models to our CI build machines. This is to ensure that our tests cover every ONNX opset.
Add dev notes
2019-02-01 22:16:30 +00:00
Fix C# and numbering (#11643) * C# protocol buffer code can be updated on Linux. Link to the relevant instructions. * Fix numbering.
2022-05-31 18:33:36 +00:00
1. Send you PR, and **manually** queue a build for every packaging pipeline for your branch.
Add dev notes
2019-02-01 22:16:30 +00:00
Support per-test tolerances for ONNX tests (#11775) Prior to this every test shared the same tolerances. This meant that if an ONNX test failed due to a small but acceptable difference in output, the only alternative was to disable the test entirely. In op set 17, the DFT operator is being added. Without this change, the tests for that operator fail because the output is off by about 5e-5. It's better to keep test coverage for this new op rather than disable the test entirely. Also prior to this change, the global tolerances were not shared between C++, JavaScript, and Python tests. Now they are. Also fix various minor issues raised by linters. Unblocks https://github.com/microsoft/onnxruntime/issues/11640.
2022-06-14 22:12:23 +00:00
1. If there is a build failure in stage "Check out of dated documents" in WebAssembly CI pipeline, update ONNX Runtime
Web WebGL operator support document:
doc: add ort-web related instructions to update onnx doc (#8500) * doc: update instructions for ort web docs * revise readme
2021-08-06 22:09:11 +00:00
- Make sure Node.js is installed (see [Prerequisites](../js/README.md#Prerequisites) for instructions).
- Follow step 1 in [js/Build](../js/README.md#Build-2) to install dependencies).
- Follow instructions in [Generate document](../js/README.md#Generating-Document) to update document. Commit changes applied to file `docs/operators.md`.
Support per-test tolerances for ONNX tests (#11775) Prior to this every test shared the same tolerances. This meant that if an ONNX test failed due to a small but acceptable difference in output, the only alternative was to disable the test entirely. In op set 17, the DFT operator is being added. Without this change, the tests for that operator fail because the output is off by about 5e-5. It's better to keep test coverage for this new op rather than disable the test entirely. Also prior to this change, the global tolerances were not shared between C++, JavaScript, and Python tests. Now they are. Also fix various minor issues raised by linters. Unblocks https://github.com/microsoft/onnxruntime/issues/11640.
2022-06-14 22:12:23 +00:00
1. Usually some newly introduced tests will fail. Then you may need to update
Add dev notes
2019-02-01 22:16:30 +00:00
- [onnxruntime/test/onnx/main.cc](/onnxruntime/test/onnx/main.cc)
Remove onnxruntime/core/protobuf (#8617) * remove onnxruntime/core/protobuf * Update How_To_Update_ONNX_Dev_Notes.md
2021-08-10 16:36:27 +00:00
- [onnxruntime/test/providers/cpu/model_tests.cc](/onnxruntime/test/providers/cpu/model_tests.cc)
- [csharp/test/Microsoft.ML.OnnxRuntime.Tests/InferenceTest.cs](/csharp/test/Microsoft.ML.OnnxRuntime.Tests/InferenceTest.cs)
move backend test filters into data file (#3798) * move backend test filters into data file * update data * update data * update document * fix list for current_failing_tests_OPENVINO_CPU_FP32
2020-05-03 02:05:58 +00:00
- [onnxruntime/test/testdata/onnx_backend_test_series_filters.jsonc](/onnxruntime/test/testdata/onnx_backend_test_series_filters.jsonc)
Support per-test tolerances for ONNX tests (#11775) Prior to this every test shared the same tolerances. This meant that if an ONNX test failed due to a small but acceptable difference in output, the only alternative was to disable the test entirely. In op set 17, the DFT operator is being added. Without this change, the tests for that operator fail because the output is off by about 5e-5. It's better to keep test coverage for this new op rather than disable the test entirely. Also prior to this change, the global tolerances were not shared between C++, JavaScript, and Python tests. Now they are. Also fix various minor issues raised by linters. Unblocks https://github.com/microsoft/onnxruntime/issues/11640.
2022-06-14 22:12:23 +00:00
- [onnxruntime/test/testdata/onnx_backend_test_series_overrides.jsonc](/onnxruntime/test/testdata/onnx_backend_test_series_overrides.jsonc)
Reference in a new issue Copy permalink