Commit graph

12 commits

Author SHA1 Message Date
Changming Sun
f9a92e589a
Upgrade the Windows SDK version that is used in WindowsAI Nuget Packaging pipeline (#19786)
### Description
1. Upgrade the version from 10.0.19041.0 to 10.0.22621.0. The old one
misses some macros that are needed by PyTorch's CPUINFO
2. Also update cmake.


### Motivation and Context
In PR #19655 I added CPUINFO to all Windows builds, but forgot to test
this pipeline.
2024-03-06 09:10:35 -08:00
Changming Sun
e91d91ae4f
Fix a build issue: /MP was not enabled correctly (#19190)
### Description

In PR #19073 I mistunderstood the value of "--parallel". Instead of
testing if args.parallel is None or not , I should test the returned
value of number_of_parallel_jobs function.

If build.py was invoked without --parallel, then args.parallel equals to
1. Because it is the default value. Then we should not add "/MP".
However, the current code adds it. Because if `args.paralllel` is
evaluated to `if 1` , which is True.
If build.py was invoked with --parallel with additional numbers, then
args.parallel equals to 0. Because it is unspecified. Then we should add
"/MP". However, the current code does not add it. Because `if
args.paralllel` is evaluated to `if 0` , which is False.

This also adds a new build flag: use_binskim_compliant_compile_flags, which is intended to be only used in ONNX Runtime team's build pipelines for compliance reasons. 

### Motivation and Context
2024-01-29 12:45:38 -08:00
Changming Sun
0e8d4c3d21
Enable Address Sanitizer in CI (#19073)
### Description
1. Add two build jobs for enabling Address Sanitizer in CI. One for
Windows CPU, One for Linux CPU.
2. Set default compiler flags/linker flags in build.py for normal
Windows/Linux/MacOS build. This can help control compiler flags in a
more centralized way.
3. All Windows binaries in our official packages will be built with
"/PROFILE" flag. Symbols of onnxruntime.dll can be found at [Microsoft
public symbol
server](https://learn.microsoft.com/en-us/windows-hardware/drivers/debugger/microsoft-public-symbols).

Limitations:
1. On Linux Address Sanitizer ignores RPATH settings in ELF binaries.
Therefore once Address Sanitizer is enabled, before running tests we
need to manually set LD_LIBRARY_PATH properly otherwise
libonnxruntime.so may not be able to find custom ops and shared EPs.
4. On Linux we also need to set LD_PRELOAD before running some tests(if
the main executable, like python, is not built with address sanitizer.
On Windows we do not need to.
5. On Windows before running python tests we should manually copy
address sanitizer DLL to the onnxruntime/capi directory, because python
3.8 and above has enabled "Safe DLL Search Mode" that wouldn't use the
information provided by PATH env.
6. On Linux Address Sanitizer found a lot of memory leaks from our
python binding code. Therefore right now we cannot enable Address
Sanitizer when building ONNX Runtime with python binding.
7. Address Sanitizer itself uses a lot of memory address space and
delays memory deallocations, which is easy to cause OOM issues in 32-bit
applications. We cannot run all the tests in onnxruntime_test_all in
32-bit mode with Address Sanitizer due to this reason. However, we still
can run individual tests in such a way. We just cannot run all of them
in one process.

### Motivation and Context
To catch memory issues.
2024-01-12 07:24:40 -08:00
Changming Sun
7047d13c68
Update windowsai-steps.yml: enable "/profile" linker flag (#18022)
### Description
Update windowsai-steps.yml: enable "/profiling" linker flag for an
internal requirement.
2023-12-13 19:47:04 -08:00
Yi Zhang
2e214d6e27
Workaround to upgrade VS2022 for Windows ARM build (#16826)
### Description



### Motivation and Context
It should be reverted when VS2022 is upgraded to 17.7 or above.

### Vefication

https://dev.azure.com/aiinfra/Lotus/_build/results?buildId=331401&view=logs&j=7517abfd-115a-5c61-78a0-7ba3c9e3a88d
2023-07-25 08:35:52 +08:00
Changming Sun
d53324d4a7
Update cmake version in a few places (#15775)
### Description
They were missed in #15707 , because they are not in common places for Dockerfiles.

Though this commit updated tools/ci_build/github/pai/rocm-ci-pipeline-env.Dockerfile, it won't automatically take effect. The image needs to be manually generated and pushed to a place, and before doing that our CMakeLists.txt also needs to be tweaked a little bit.
2023-05-02 22:56:28 -07:00
Sheil Kumar
0fbbb6a43e
WindowsAI build failing due to deprecated .NET5 SDK missing in build image (#15383)
WindowsAI build failing due to deprecated .NET5 SDK missing in build
image

.NET5 was deprecated last year, and recently the build machine images
have been updated to not include this SDK.
Unblock failing builds by force insalling .NET5 SDK as part of the build
pipeline.
2023-04-06 08:51:07 -07:00
Yi Zhang
cad7ef93e6
use python 3.9.7 in windowai packaging pipeline (#14766)
### Description
Use python3.9.7 in windowsAI packaging pipeline.


### Motivation and Context
In WindowsAI packaging pipeline, cdpxwin1809 is deprecated and it will
no longer be cached from March 2023.

I used the recommended image
[onebranch.azurecr.io/windows/ltsc2019/vse2022:latest](https://onebranch.visualstudio.com/OneBranch/_wiki/wikis/OneBranch.wiki/4587/Container-Images?anchor=recommendation-for-windows-container-image-userst)
But it always failed to pass arm32 jobs. 
It's very likely a regression in VS2022. 
One user reported a similar issue #14190. 
I've submitted a bug to visual studio.
https://developercommunity.visualstudio.com/t/Compilation-Error-with-VS2022-ARM/10285309.

For `onebranch.azurecr.io/windows/ltsc2019/vse2019:latest`, there's an
exception` Error : init_sys_streams: can't initialize sys standard
streams`
It could be solved by updating from python3.7 to python3.9 in the
pipeline.

(https://dev.azure.com/aiinfra/Lotus/_build/results?buildId=279301&view=results)
The root cause might be the conflicts between different python version.
The inherent python in onebranch image is python 3.9.7.

(https://onebranch.visualstudio.com/OneBranch/_wiki/wikis/OneBranch.Code.Wiki/6766/manifest)


### Ref

https://onebranch.visualstudio.com/OneBranch/_wiki/wikis/OneBranch.wiki/4587/Container-Images?anchor=windows-build-images
2023-02-23 09:48:42 +08:00
Yi Zhang
8ed3dfe063
Revert "try VS 2022 in windowsAI pipeline (#14608)" (#14619)
This reverts commit f88a4646cd.

### Description
<!-- Describe your changes. -->



### Motivation and Context
For release, winai packaing pipeline's container image is revert to old
image.
So we should revert VS to 2019
2023-02-08 12:45:37 +08:00
Yi Zhang
f88a4646cd
try VS 2022 in windowsAI pipeline (#14608)
### Description
update VS2019 to VS 2022 in
onnxruntime-Nuget-WindowsAI-Pipeline-Official


### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
2023-02-07 17:53:53 +08:00
Changming Sun
efcbdac58e
Remove the cmake option: onnxruntime_DEV_MODE (#13573)
1. Remove the cmake option onnxruntime_DEV_MODE and replace it with
"--compile-no-warning-as-error"
2. Suppress some GSL warnings because now we treat nvcc diag warnings as
errors
2022-11-07 09:06:28 -08:00
Changming Sun
b3e96d6195
A new pipeline to replace the existing WindowsAI packaging pipeline (#10646) 2022-03-03 08:56:49 -08:00