Changming Sun
5f6a3158f8
Enable VCPKG in CI build ( #23426 )
...
### Description
1. Enable VCPKG flag in Windows CPU CI build pipelines.
2. Increased the min supported cmake version from 3.26 to 3.28. Because
of it, drop the support for the old way of finding python by
"find_package(PythonLibs)". Therefore, in build.py we no longer set
"PYTHON_EXECUTABLE" cmake var when doing cmake configure.
3. Added "xnnpack-ep" as a feature for ORT's vcpkg config.
4. Added asset cache support for ORT's vcpkg build
5. Added VCPKG triplet files for Android build.
6. Set VCPKG triplet to "universal2-osx" if CMAKE_OSX_ARCHITECTURES was
found in cmake extra defines.
7. Removed a small piece of code in build.py, which was for support CUDA
version < 11.8.
8. Fixed an issue that CMAKE_OSX_ARCHITECTURES sometimes got specified
twice when build.py invoked cmake.
9. Added more model tests to Android build. After this change, we will
test all ONNX versions instead of just the latest one.
10. Fixed issues that are related to build.py's "--build_nuget"
parameter. Also, enable the flag in most Windows CPU CI build jobs.
11. Removed a restriction in build.py that disallowed cross-compiling
Windows ARM64 nuget package on Windows x86.
### Motivation and Context
Adopt vcpkg.
2025-02-05 10:58:53 -08:00
PARK DongHa
169917b1e7
Use latest vcpkg commit in configuration, sync manifest with deps.txt ( #23554 )
...
### Description
`python3` dependency is removed in `onnx` port of
https://github.com/microsoft/vcpkg upstream.
* https://github.com/microsoft/vcpkg/pull/43236
*
https://github.com/microsoft/onnxruntime/pull/23285#issuecomment-2579073056
(Previous work)
Removed `nsync`, and use ONNX 1.70.0+ in vcpkg.json(manifest)
### Motivation and Context
* Help #23158
* #23456
2025-01-31 12:34:07 -08:00
PARK DongHa
5b9c968eaa
Correct ONNX and Protobuf version in vcpkg build ( #23285 )
...
### Description
Changes vcpkg manifest and configuration file (vcpkg.json &
vcpkg-configuration.json)
* Update vcpkg version to
https://github.com/microsoft/vcpkg/releases/tag/2024.12.16
* Use protobuf 3.21.12(= `v21.12`) to sync with
[cmake/deps.txt](https://github.com/microsoft/onnxruntime/blob/main/cmake/deps.txt )
* Resolve https://github.com/microsoft/onnxruntime/issues/22750
* Add `onnx` to vcpkg manifest so `find_package(ONNX)` and
`find_dependency(Protobuf)` can work as expected.
* Currently, It uses 1.16.2
* v1.17.0 will become available after
https://github.com/microsoft/vcpkg/pull/42942
However, `onnx` in vcpkg doesn't configure
`ONNX_DISABLE_STATIC_REGISTRATION` build option.
* https://github.com/microsoft/vcpkg/pull/38879
* Create "cmake/vcpkg-triplets/" folder and triplet files which use
`VCPKG_CMAKE_CONFIGURE_OPTIONS` for the option
* This requires `VCPKG_OVERLAY_TRIPLETS` environment variable for CI
steps, which is a bit inconvenient.
I will try to find simple way to get same result
### Motivation and Context
* Help #23158
* "ONNX is not consumed from vcpkg"
* "Mismatch protobuf version. When vcpkg is enabled , we should not
fetch protoc from Github which may cause version mismatches."
* https://github.com/microsoft/vcpkg/pull/43126
* #21348
2025-01-08 12:25:17 -08:00
Misha Chornyi
bf4d3e1a5b
Update vcpkg.json - lock flatbuffer version ( #23046 )
...
### Description
Locking version introduced in:
03ea5dc495/onnxruntime/core/flatbuffers/schema/ort_training_checkpoint.fbs.h (L11-L13)https://github.com/microsoft/onnxruntime/issues/22666
2024-12-10 11:23:01 -08:00
PARK DongHa
f633caa0b1
Create CMake option onnxruntime_USE_VCPKG ( #21348 )
...
### Changes
1. CMake option `onnxruntime_USE_VCPKG`. It will be used in the vcpkg
port
* Unit test may fail because this option leads to a mixture of
unexpected external library versions.
Especially ONNX, Protobuf, and Flatbuffers version can be different
2. Overhaul of `onnxruntime_external_deps.cmake`
* Make `FetchContent_Declare` to try `find_package`.
See
https://cmake.org/cmake/help/latest/guide/using-dependencies/index.html
* Relocated `FetchContent_Declare` and `FetchContent_MakeAvailable`(or
`onnxruntime_fetchcontent_makeavailable`) to closer lines.
It was too hard to navigate the entire file to search related
sections...
* Alias `IMPORTED` targets like build targets (e.g. `ONNX::onnx` -->
`onnx`)
```cmake
# The script uses `find_package` with the changes.
# In this case, use vcpkg to search dependencies
# See https://cmake.org/cmake/help/latest/guide/using-dependencies/index.html
include(external/onnxruntime_external_deps.cmake)
```
3. Create CMakePresets.json and presets to [run vcpkg in manifest
mode](https://learn.microsoft.com/en-us/vcpkg/concepts/manifest-mode )
* Currently, it's NOT for training build
* Main triplets are `x64-windows` and `x64-osx`
```pwsh
Push-Location "cmake"
cmake --preset "x64-windows-vcpkg"
cmake --build --preset "x64-windows-vcpkg-debug"
Pop-Location
```
```bash
pushd "cmake"
cmake --preset "x64-osx-vcpkg"
cmake --build --preset "x64-osx-vcpkg-debug"
popd
```
4. Updated tools/ci_build/build.py
* `--use_vcpkg` option: it needs `CMAKE_TOOLCHAIN_FILE` with
[vcpkg.cmake toolchain
script](https://github.com/microsoft/vcpkg/blob/master/scripts/buildsystems/vcpkg.cmake )
* `--compile_no_warning_as_error` is recommended because library version
differences will cause unexpected compiler warnings
```bash
python ./tools/ci_build/build.py \
--compile_no_warning_as_error \
--use_vcpkg \
--cmake_extra_defines "CMAKE_TOOLCHAIN_FILE:FILEPATH=${VCPKG_ROOT}/scripts/buildsystems/vcpkg.cmake" \
--cmake_extra_defines "VCPKG_TARGET_TRIPLET=..."
```
5. Created Job `Vcpkg` for Windows and macOS
* Show how to setup and use vcpkg.
Similar to the CMakePresets.json usage
### Motivation and Context
* Help #7150
* Help https://github.com/microsoft/vcpkg/pull/36850
* https://github.com/luncliff/vcpkg-registry/pull/212
* https://github.com/microsoft/vcpkg/pull/39881
* https://github.com/luncliff/vcpkg-registry/pull/215
* https://github.com/luncliff/vcpkg-registry/pull/216
* https://github.com/luncliff/vcpkg-registry/pull/227
*
https://cmake.org/cmake/help/latest/guide/using-dependencies/index.html
*
https://github.com/microsoft/vcpkg/blob/master/scripts/buildsystems/vcpkg.cmake
### Future Works?
More feature coverage with the vcpkg supported libraries
* CUDA feature support
* Training feature support
2024-09-10 16:39:27 -07:00
