onnxruntime/tools
Yi Zhang 87a9f77c56
Refactor Python Packaing Pipeline (Training Cuda 11.8) (#19910)
### Description
1. Use stage to organize the pipeline and split building and testing
2. Move compilation on CPU machine
3. test stage can leverage existing artifacts
4. check wheel size, it gives warning if the size above 300M
5. docker image name wasn't change even the argument changed, which
caused the docker image was always rebuilt. So update the docker image
name according to the argument can save the docker build time.

Pipeline duration reduced by 60% (2 hours ->  50 minutes)
Compilation time reduced by 75% (1.5hours -> 20 minutes)
GPU time reduced by 87% ( 8 hours to 1 hours)
for debugging, the GPU time could be reduced by above 95%, because we
can choose run only one test stage and skip building.

### Motivation and Context
Make the pipeline efficient.
Optimized

https://dev.azure.com/aiinfra/Lotus/_build/results?buildId=424177&view=results
Curent

https://dev.azure.com/aiinfra/Lotus/_build/results?buildId=422393&view=results

---------
2024-03-15 06:47:41 +08:00
..
android_custom_build Update NDK version to 26.1.10909125 (#18493) 2023-11-17 14:14:01 -08:00
ci_build Refactor Python Packaing Pipeline (Training Cuda 11.8) (#19910) 2024-03-15 06:47:41 +08:00
doc Bump ruff to 0.3.2 and black to 24 (#19878) 2024-03-13 10:00:32 -07:00
nuget Bump ruff to 0.3.2 and black to 24 (#19878) 2024-03-13 10:00:32 -07:00
perf_view fixed #16873 (#16932) 2023-09-26 09:57:01 -07:00
python Bump ruff to 0.3.2 and black to 24 (#19878) 2024-03-13 10:00:32 -07:00
scripts Fix a build issue: /MP was not enabled correctly (#19190) 2024-01-29 12:45:38 -08:00