onnxruntime/tools/ci_build/github/azure-pipelines
Yi Zhang c5f5e3ec5e
Improve 2 cache tasks in one pipeline yaml (#15267)
### Description
1. Make 2 cache tasks in one pipeline really works
2. Each building step has its own environment variable CCACHE_DIR
instead of job variables.
3. Extenal Protobuf compilation cache only updates with deps.txt. It
doesn't generate new cache in every commit.


### Motivation and Context
The simple workflow is as below
```
--------build with ccache-------             
         |                       
        cache                    
         |                       
      {CCACHE_DIR}-----cache stat.
```

```
-------Cache@2------
           |
    download cache           
           |                         
          {path}--------upload cache
```

1. {XXX} means environment variable or task input.
2. {CCACHE_DIR} must be consistent with {path}. Ccache produces caches
in {CCACHE_DIR} and Cache@2 download cache into {path} and tar {path}
and upload it.
3. Protobuf changes with deps.txt so that it would reduce the storage
size.
4. Next step, we may split the compilation into 2 steps, one for
external dependencies and another for ORT.
2023-03-30 23:22:11 +08:00
..
nodejs/templates Cjian/multi stage packaging pipeline (#14993) 2023-03-24 23:39:15 -07:00
nuget/templates Cjian/windows update python3.11 (#15243) 2023-03-28 22:15:47 -07:00
templates Improve 2 cache tasks in one pipeline yaml (#15267) 2023-03-30 23:22:11 +08:00
android-arm64-v8a-QNN-crosscompile-ci-pipeline.yml Remove protobuf submodule (#15190) 2023-03-27 10:35:49 -07:00
android-x86_64-crosscompile-ci-pipeline.yml Move Linux CPU pipelines to an AMD CPU pool which is cheaper (#15144) 2023-03-27 14:10:08 -07:00
anybuild.yml
binary-size-checks-pipeline.yml Update python 3.11 and remove 3.7 for Linux (#15214) 2023-03-27 14:46:30 -07:00
build-perf-test-binaries-pipeline.yml Update python 3.11 and remove 3.7 for Linux (#15214) 2023-03-27 14:46:30 -07:00
c-api-noopenmp-packaging-pipelines.yml Update the post-merge pipeline (#14965) 2023-03-29 13:12:07 -07:00
clean-build-docker-image-cache-pipeline.yml
linux-ci-pipeline.yml Update python 3.11 and remove 3.7 for Linux (#15214) 2023-03-27 14:46:30 -07:00
linux-cpu-aten-pipeline.yml Update python 3.11 and remove 3.7 for Linux (#15214) 2023-03-27 14:46:30 -07:00
linux-cpu-eager-pipeline.yml Update python 3.11 and remove 3.7 for Linux (#15214) 2023-03-27 14:46:30 -07:00
linux-cpu-minimal-build-ci-pipeline.yml Update python 3.11 and remove 3.7 for Linux (#15214) 2023-03-27 14:46:30 -07:00
linux-dnnl-ci-pipeline.yml Update python 3.11 and remove 3.7 for Linux (#15214) 2023-03-27 14:46:30 -07:00
linux-gpu-ci-pipeline.yml Update python 3.11 and remove 3.7 for Linux (#15214) 2023-03-27 14:46:30 -07:00
linux-gpu-tensorrt-ci-pipeline.yml Update python 3.11 and remove 3.7 for Linux (#15214) 2023-03-27 14:46:30 -07:00
linux-gpu-tensorrt-daily-perf-pipeline.yml
linux-migraphx-ci-pipeline.yml
linux-multi-gpu-ci-pipeline.yml Update python 3.11 and remove 3.7 for Linux (#15214) 2023-03-27 14:46:30 -07:00
linux-multi-gpu-tensorrt-ci-pipeline.yml
linux-openvino-ci-pipeline.yml
linux-openvino-nightly-pipeline.yml
linux-qnn-ci-pipeline.yml [QNN EP] Update QNN SDK to 2.8 (#14978) 2023-03-10 13:21:19 -08:00
mac-ci-pipeline.yml Cjian/multi stage packaging pipeline (#14993) 2023-03-24 23:39:15 -07:00
mac-coreml-ci-pipeline.yml Add Compliation Cache in CoreML pipeline (#15259) 2023-03-30 23:18:52 +08:00
mac-ios-ci-pipeline.yml Add compilation cache in iOS pipeline (#15070) 2023-03-16 21:43:18 +08:00
mac-ios-packaging-pipeline.yml Update to MacOS-12 (#14924) 2023-03-09 10:18:14 -08:00
mac-objc-static-analysis-ci-pipeline.yml Update clang-tidy path for updated Mac image. (#14760) 2023-02-22 09:00:42 -08:00
mac-react-native-ci-pipeline.yml Update python 3.11 and remove 3.7 for Linux (#15214) 2023-03-27 14:46:30 -07:00
npm-packaging-pipeline.yml Update python 3.11 and remove 3.7 for Linux (#15214) 2023-03-27 14:46:30 -07:00
orttraining-linux-ci-pipeline.yml Update python 3.11 and remove 3.7 for Linux (#15214) 2023-03-27 14:46:30 -07:00
orttraining-linux-external-custom-ops.yml Update python 3.11 and remove 3.7 for Linux (#15214) 2023-03-27 14:46:30 -07:00
orttraining-linux-gpu-amd-e2e-test-ci-pipeline.yml
orttraining-linux-gpu-ci-pipeline.yml
orttraining-linux-gpu-distributed-e2e-test-pipeline.yml
orttraining-linux-gpu-docker-release-pipeline.yml
orttraining-linux-gpu-ortmodule-distributed-test-ci-pipeline.yml Introduce collective ops to ort inference build (#14399) 2023-02-07 13:47:48 -08:00
orttraining-linux-gpu-ortmodule-test-clear-cache-pipeline.yml Move Linux CPU pipelines to an AMD CPU pool which is cheaper (#15144) 2023-03-27 14:10:08 -07:00
orttraining-linux-gpu-training-apis.yml [Java] Initial on device training support (#14027) 2023-03-08 10:01:08 -08:00
orttraining-linux-nightly-ortmodule-test-pipeline.yml Update acpt image in the training pipeline (#14855) 2023-03-07 14:10:32 -08:00
orttraining-mac-ci-pipeline.yml Cjian/multi stage packaging pipeline (#14993) 2023-03-24 23:39:15 -07:00
orttraining-pai-ci-pipeline.yml Enable ccache for HIP objects (#14465) 2023-01-28 22:34:24 +08:00
orttraining-py-packaging-pipeline-cpu.yml Update python 3.11 and remove 3.7 for Linux (#15214) 2023-03-27 14:46:30 -07:00
orttraining-py-packaging-pipeline-cuda116.yml Update python 3.11 and remove 3.7 for Linux (#15214) 2023-03-27 14:46:30 -07:00
orttraining-py-packaging-pipeline-rocm.yml [ROCm] remove python3.7 from python packaging pipeline (#15230) 2023-03-28 10:37:04 +08:00
post-merge-jobs.yml Update the post-merge pipeline (#14965) 2023-03-29 13:12:07 -07:00
py-package-build-pipeline.yml Update python 3.11 and remove 3.7 for Linux (#15214) 2023-03-27 14:46:30 -07:00
py-package-test-pipeline.yml Add smoking tests for all CPU Packages. (#15153) 2023-03-24 12:30:05 +08:00
py-packaging-pipeline.yml Update python 3.11 and remove 3.7 for Linux (#15214) 2023-03-27 14:46:30 -07:00
sign_ov_ep_binaries.yml
snpe-ep-nuget-packaging-pipeline.yml Cjian/windows update python3.11 (#15243) 2023-03-28 22:15:47 -07:00
web-ci-pipeline.yml Move Linux CPU pipelines to an AMD CPU pool which is cheaper (#15144) 2023-03-27 14:10:08 -07:00
web-packaging-pipeline.yml
win-ci-fuzz-testing.yml Cjian/windows update python3.11 (#15243) 2023-03-28 22:15:47 -07:00
win-ci-pipeline.yml Remove intermedia obj files once build finished (#14361) 2023-01-20 13:37:15 +08:00
win-eager-ci-pipeline.yml
win-gpu-ci-pipeline.yml
win-gpu-reduce-op-ci-pipeline.yml Cjian/windows update python3.11 (#15243) 2023-03-28 22:15:47 -07:00
win-gpu-tensorrt-ci-pipeline.yml Cjian/windows update python3.11 (#15243) 2023-03-28 22:15:47 -07:00
win-qnn-arm64-ci-pipeline.yml Update protobuf to 3.21.x (#15245) 2023-03-29 14:08:18 -07:00
win-qnn-ci-pipeline.yml Cjian/windows update python3.11 (#15243) 2023-03-28 22:15:47 -07:00