mirror of
https://github.com/saymrwulf/onnxruntime.git
synced 2026-05-28 22:56:32 +00:00
8818a99c93
### Description There are 8 cu files under [flash attention](https://github.com/microsoft/onnxruntime/tree/main/onnxruntime/contrib_ops/cuda/bert/flash_attention) and 4 cu files under [cutlass fmha](https://github.com/microsoft/onnxruntime/tree/main/onnxruntime/contrib_ops/cuda/bert/cutlass_fmha) need a lot of memory to compile. Previously, the default value is same as parallel - number of CPU cores. Standard_NC4as_T4_v3 has 4 CPUs and 28 GB memory, and we launched 16 nvcc threads in total (4 parallel jobs, and 4 nvcc threads per job). Each thread might take 4 GB on average (peak is around 6GB, but threads are not started at same time). OOM happens since 16 threads might need close to 64 GB in worst case. When build machine has 64GB or larger memory, OOM is rare. Here we set a proper nvcc --threads based on available memory to avoid OOM. ### Motivation and Context Fix `Python Packaging Pipeline (Training Cuda 11.8)` |
||
|---|---|---|
| .. | ||
| github | ||
| __init__.py | ||
| amd_hipify.py | ||
| build.py | ||
| clean_docker_image_cache.py | ||
| compile_triton.py | ||
| coverage.py | ||
| gen_def.py | ||
| get_docker_image.py | ||
| logger.py | ||
| op_registration_utils.py | ||
| op_registration_validator.py | ||
| patch_manylinux.py | ||
| policheck_exclusions.xml | ||
| reduce_op_kernels.py | ||
| replace_urls_in_deps.py | ||
| requirements.txt | ||
| set-trigger-rules.py | ||
| update_tsaoptions.py | ||
| upload_python_package_to_azure_storage.py | ||