onnxruntime/tools/ci_build/github/azure-pipelines
Changming Sun 0e8d4c3d21
Enable Address Sanitizer in CI (#19073)
### Description
1. Add two build jobs for enabling Address Sanitizer in CI. One for
Windows CPU, One for Linux CPU.
2. Set default compiler flags/linker flags in build.py for normal
Windows/Linux/MacOS build. This can help control compiler flags in a
more centralized way.
3. All Windows binaries in our official packages will be built with
"/PROFILE" flag. Symbols of onnxruntime.dll can be found at [Microsoft
public symbol
server](https://learn.microsoft.com/en-us/windows-hardware/drivers/debugger/microsoft-public-symbols).

Limitations:
1. On Linux Address Sanitizer ignores RPATH settings in ELF binaries.
Therefore once Address Sanitizer is enabled, before running tests we
need to manually set LD_LIBRARY_PATH properly otherwise
libonnxruntime.so may not be able to find custom ops and shared EPs.
4. On Linux we also need to set LD_PRELOAD before running some tests(if
the main executable, like python, is not built with address sanitizer.
On Windows we do not need to.
5. On Windows before running python tests we should manually copy
address sanitizer DLL to the onnxruntime/capi directory, because python
3.8 and above has enabled "Safe DLL Search Mode" that wouldn't use the
information provided by PATH env.
6. On Linux Address Sanitizer found a lot of memory leaks from our
python binding code. Therefore right now we cannot enable Address
Sanitizer when building ONNX Runtime with python binding.
7. Address Sanitizer itself uses a lot of memory address space and
delays memory deallocations, which is easy to cause OOM issues in 32-bit
applications. We cannot run all the tests in onnxruntime_test_all in
32-bit mode with Address Sanitizer due to this reason. However, we still
can run individual tests in such a way. We just cannot run all of them
in one process.

### Motivation and Context
To catch memory issues.
2024-01-12 07:24:40 -08:00
..
nodejs/templates Move a nodejs test to a different machine pool (#17970) 2023-10-17 09:30:14 -07:00
nuget/templates Fix Nuget CUDA Packaging pipeline (#19054) 2024-01-11 11:59:21 -08:00
stages Fix Nuget CUDA Packaging pipeline (#19054) 2024-01-11 11:59:21 -08:00
templates Enable Address Sanitizer in CI (#19073) 2024-01-12 07:24:40 -08:00
triggers
android-arm64-v8a-QNN-crosscompile-ci-pipeline.yml [QNN EP] Update QNN SDK to version 2.17.0 (#18684) 2023-12-06 11:05:41 -08:00
android-x86_64-crosscompile-ci-pipeline.yml Attempt to make the usage of the Android emulator in CIs more robust (#17903) 2023-10-15 08:42:36 +10:00
binary-size-checks-pipeline.yml
build-perf-test-binaries-pipeline.yml
c-api-noopenmp-packaging-pipelines.yml Enable Address Sanitizer in CI (#19073) 2024-01-12 07:24:40 -08:00
clean-build-docker-image-cache-pipeline.yml
cuda-packaging-pipeline.yml Fix Nuget CUDA Packaging pipeline (#19054) 2024-01-11 11:59:21 -08:00
linux-ci-pipeline.yml Enable Address Sanitizer in CI (#19073) 2024-01-12 07:24:40 -08:00
linux-cpu-aten-pipeline.yml Update nodejs to 18.x (#17657) 2023-09-25 14:12:11 -07:00
linux-cpu-eager-pipeline.yml
linux-cpu-minimal-build-ci-pipeline.yml Set NDK version in Linux CPU Minimal Build E2E CI Pipeline (#18810) 2023-12-14 08:08:41 -08:00
linux-dnnl-ci-pipeline.yml
linux-gpu-ci-pipeline.yml Create a new Nuget Package pipeline for CUDA 12 (#18135) 2023-11-28 09:03:46 -08:00
linux-gpu-tensorrt-ci-pipeline.yml Create a new Nuget Package pipeline for CUDA 12 (#18135) 2023-11-28 09:03:46 -08:00
linux-gpu-tensorrt-daily-perf-pipeline.yml [EP Perf] Fix missing Azure cli & use onnx zoo model inside image (#18917) 2024-01-01 17:14:39 -08:00
linux-migraphx-ci-pipeline.yml [ROCm] Update CI/Packaging pipeline to ROCm6.0 (#18985) 2024-01-03 17:25:15 +08:00
linux-multi-gpu-tensorrt-ci-pipeline.yml Update nodejs to 18.x (#17657) 2023-09-25 14:12:11 -07:00
linux-openvino-ci-pipeline.yml Update nodejs to 18.x (#17657) 2023-09-25 14:12:11 -07:00
linux-qnn-ci-pipeline.yml [QNN EP] Support multithreaded inference of a single session (#18981) 2024-01-04 13:32:48 -08:00
mac-ci-pipeline.yml
mac-coreml-ci-pipeline.yml Update min macos version (#18251) 2023-11-10 11:08:17 -08:00
mac-ios-ci-pipeline.yml Enable Address Sanitizer in CI (#19073) 2024-01-12 07:24:40 -08:00
mac-ios-packaging-pipeline.yml Remove swift files on ORT main repo (#17799) 2023-10-05 15:27:15 -07:00
mac-objc-static-analysis-ci-pipeline.yml Update absl and gtest to fix an ARM64EC build error (#18735) 2023-12-07 15:55:17 -08:00
mac-react-native-ci-pipeline.yml
npm-packaging-pipeline.yml use EO pool for windows web_cpu stage (#18737) 2023-12-07 10:10:00 -08:00
nuget-cuda-publishing-pipeline.yml Update Nuget publishing jobs (#18851) 2023-12-19 16:54:46 -08:00
orttraining-linux-ci-pipeline.yml Update nodejs to 18.x (#17657) 2023-09-25 14:12:11 -07:00
orttraining-linux-gpu-ci-pipeline.yml Update nodejs to 18.x (#17657) 2023-09-25 14:12:11 -07:00
orttraining-linux-gpu-ortmodule-distributed-test-ci-pipeline.yml ONNX-Native Tensor Parallel: Using Distributed MatMul as Example (#17695) 2023-10-05 14:22:25 -07:00
orttraining-linux-nightly-ortmodule-test-pipeline.yml
orttraining-mac-ci-pipeline.yml
orttraining-pai-ci-pipeline.yml [ROCm] Update CI/Packaging pipeline to ROCm6.0 (#18985) 2024-01-03 17:25:15 +08:00
orttraining-py-packaging-pipeline-cpu.yml Adding python3.12 support to ORT (#18814) 2024-01-11 08:34:28 -08:00
orttraining-py-packaging-pipeline-cuda.yml Updates to training pipelines to reduce CI time (#18116) 2023-10-26 14:58:57 -07:00
orttraining-py-packaging-pipeline-cuda12.yml Training packaging pipeline for cuda12 (#18524) 2023-11-21 13:19:21 -08:00
orttraining-py-packaging-pipeline-rocm.yml [ROCm] Update CI/Packaging pipeline to ROCm6.0 (#18985) 2024-01-03 17:25:15 +08:00
post-merge-jobs.yml Enable Address Sanitizer in CI (#19073) 2024-01-12 07:24:40 -08:00
publish-nuget.yml Update Nuget publishing jobs (#18851) 2023-12-19 16:54:46 -08:00
py-cuda-package-test-pipeline.yml Adding new pipeline for python cuda testing (#18718) 2023-12-18 18:13:03 -08:00
py-cuda-packaging-pipeline.yml Update the template files to correct stage to fix the python cuda 12 packaging pipeline (#18651) 2023-12-01 07:57:46 -08:00
py-cuda-publishing-pipeline.yml Adding a new pipeline for publishing to Python Cuda 12 packages. (#18712) 2023-12-11 14:17:46 -08:00
py-package-build-pipeline.yml
py-package-test-pipeline.yml Replace all Azure-Pipelines-EO-Windows2022-aiinfrat to Onnxruntime-Win-CPU-2022 (#18614) 2023-11-29 10:32:42 -08:00
py-packaging-pipeline.yml Enable Address Sanitizer in CI (#19073) 2024-01-12 07:24:40 -08:00
qnn-ep-nuget-packaging-pipeline.yml [QNN Nuget Pipeline] Build with ML ops and detect ORT version (#19024) 2024-01-08 12:44:12 -08:00
web-ci-pipeline.yml
win-ci-fuzz-testing.yml [Fix] exception in Fuzz Test pipeline (#18984) 2024-01-03 14:53:31 +08:00
win-ci-pipeline.yml Enable Address Sanitizer in CI (#19073) 2024-01-12 07:24:40 -08:00
win-gpu-ci-pipeline.yml Move Windows GPU training job to A10 (#19041) 2024-01-08 09:19:58 -08:00
win-gpu-reduce-op-ci-pipeline.yml Enable Address Sanitizer in CI (#19073) 2024-01-12 07:24:40 -08:00
win-gpu-tensorrt-ci-pipeline.yml Enable Address Sanitizer in CI (#19073) 2024-01-12 07:24:40 -08:00
win-qnn-arm64-ci-pipeline.yml [QNN EP] Support multithreaded inference of a single session (#18981) 2024-01-04 13:32:48 -08:00
win-qnn-ci-pipeline.yml [QNN EP] Support multithreaded inference of a single session (#18981) 2024-01-04 13:32:48 -08:00