From 31f7cfb09d78edf8d444200ebec5ebe83f84828b Mon Sep 17 00:00:00 2001 From: Tianlei Wu Date: Wed, 17 Jul 2024 10:44:21 -0700 Subject: [PATCH] [Doc] cuda and cudnn requirements for 1.18.1 gpu packages (#21377) ### Description Add CUDA and cuDNN requirements for 1.18.1 GPU package. ### Motivation and Context https://github.com/microsoft/onnxruntime/issues/21354 https://github.com/microsoft/onnxruntime/issues/21173 --- .../CUDA-ExecutionProvider.md | 52 ++++++++++++------- 1 file changed, 33 insertions(+), 19 deletions(-) diff --git a/docs/execution-providers/CUDA-ExecutionProvider.md b/docs/execution-providers/CUDA-ExecutionProvider.md index 11dc4c8a94..97374ff6e0 100644 --- a/docs/execution-providers/CUDA-ExecutionProvider.md +++ b/docs/execution-providers/CUDA-ExecutionProvider.md @@ -29,28 +29,42 @@ See [Build instructions](../build/eps.html#cuda). ## Requirements Please reference table below for official GPU packages dependencies for the ONNX Runtime inferencing package. Note that -ONNX Runtime Training is aligned with PyTorch CUDA versions; refer to the Training tab -on [onnxruntime.ai](https://onnxruntime.ai/) for supported versions. +ONNX Runtime Training is aligned with PyTorch CUDA versions; refer to the Optimize Training tab on [onnxruntime.ai](https://onnxruntime.ai/getting-started) for supported versions. -Note: Because of [Nvidia CUDA Minor Version Compatibility](https://docs.nvidia.com/deploy/cuda-compatibility/#minor-version-compatibility), ONNX Runtime built with CUDA 11.8 should be compatible with any CUDA 11.x version; ONNX Runtime built with CUDA 12.2 should be compatible with any CUDA 12.x version. +Because of [Nvidia CUDA Minor Version Compatibility](https://docs.nvidia.com/deploy/cuda-compatibility/#minor-version-compatibility), ONNX Runtime built with CUDA 11.8 are compatible with any CUDA 11.x version; ONNX Runtime built with CUDA 12.x are compatible with any CUDA 12.x version. -ONNX Runtime built with cuDNN 8.x are not compatible with cuDNN 9.x. +ONNX Runtime built with cuDNN 8.x is not compatible with cuDNN 9.x, and vice versa. You can choose the package based on CUDA and cuDNN major versions that match your runtime environment (For example, PyTorch 2.3 uses cuDNN 8.x, while PyTorch 2.4 or later used cuDNN 9.x). -| ONNX Runtime | CUDA | cuDNN | Notes | -|--------------------------|--------|-----------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| 1.18 | 12.4 | 8.9.2.26 (Linux)
8.9.2.26 (Windows) | The default CUDA version for ORT 1.18 is CUDA 11.8. To install CUDA 12 package, please look at [Install ORT](../install). Java CUDA 12 support is back for release 1.18 | -| 1.18 | 11.8 | 8.9.2.26 (Linux)
8.9.2.26 (Windows) | | -| 1.17 | 12.2 | 8.9.2.26 (Linux)
8.9.2.26 (Windows) | The default CUDA version for ORT 1.17 is CUDA 11.8. To install CUDA 12 package, please look at [Install ORT](../install).
Due to low demand on Java GPU package, only C++/C# Nuget and Python packages are released with CUDA 12.2 | -| 1.15
1.16
1.17 | 11.8 | 8.2.4 (Linux)
8.5.0.96 (Windows) | Tested with CUDA versions from 11.6 up to 11.8, and cuDNN from 8.2.4 up to 8.9.0 | -| 1.14
1.13.1
1.13 | 11.6 | 8.2.4 (Linux)
8.5.0.96 (Windows) | libcudart 11.4.43
libcufft 10.5.2.100
libcurand 10.2.5.120
libcublasLt 11.6.5.2
libcublas 11.6.5.2
libcudnn 8.2.4 | -| 1.12
1.11 | 11.4 | 8.2.4 (Linux)
8.2.2.26 (Windows) | libcudart 11.4.43
libcufft 10.5.2.100
libcurand 10.2.5.120
libcublasLt 11.6.5.2
libcublas 11.6.5.2
libcudnn 8.2.4 | -| 1.10 | 11.4 | 8.2.4 (Linux)
8.2.2.26 (Windows) | libcudart 11.4.43
libcufft 10.5.2.100
libcurand 10.2.5.120
libcublasLt 11.6.1.51
libcublas 11.6.1.51
libcudnn 8.2.4 | -| 1.9 | 11.4 | 8.2.4 (Linux)
8.2.2.26 (Windows) | libcudart 11.4.43
libcufft 10.5.2.100
libcurand 10.2.5.120
libcublasLt 11.6.1.51
libcublas 11.6.1.51
libcudnn 8.2.4 | -| 1.8 | 11.0.3 | 8.0.4 (Linux)
8.0.2.39 (Windows) | libcudart 11.0.221
libcufft 10.2.1.245
libcurand 10.2.1.245
libcublasLt 11.2.0.252
libcublas 11.2.0.252
libcudnn 8.0.4 | -| 1.7 | 11.0.3 | 8.0.4 (Linux)
8.0.2.39 (Windows) | libcudart 11.0.221
libcufft 10.2.1.245
libcurand 10.2.1.245
libcublasLt 11.2.0.252
libcublas 11.2.0.252
libcudnn 8.0.4 | -| 1.5-1.6 | 10.2 | 8.0.3 | CUDA 11 can be built from source | -| 1.2-1.4 | 10.1 | 7.6.5 | Requires cublas10-10.2.1.243; cublas 10.1.x will not work | -| 1.0-1.1 | 10.0 | 7.6.4 | CUDA versions from 9.1 up to 10.1, and cuDNN versions from 7.1 up to 7.4 should also work with Visual Studio 2017 | +### CUDA 12.x + +To install CUDA 12 package, please look at [Install ORT](../install). + +| ONNX Runtime | CUDA | cuDNN | Notes | +|---------------|--------|-------|----------------------------------------------------------------------| +| 1.18.1 | 12.x | 9.x | cuDNN 9 is required. No Java package. | +| 1.18.0 | 12.x | 8.x | Java package is added. | +| 1.17.x | 12.x | 8.x | Only C++/C# Nuget and Python packages are released. No Java package. | + +### CUDA 11.x + +| ONNX Runtime | CUDA | cuDNN | Notes | +|----------------------|--------|-----------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------| +| 1.18.x | 11.8 | 8.x | | +| 1.17
1.16
1.15 | 11.8 | 8.2.4 (Linux)
8.5.0.96 (Windows) | Tested with CUDA versions from 11.6 up to 11.8, and cuDNN from 8.2 up to 8.9 | +| 1.14
1.13 | 11.6 | 8.2.4 (Linux)
8.5.0.96 (Windows) | libcudart 11.4.43
libcufft 10.5.2.100
libcurand 10.2.5.120
libcublasLt 11.6.5.2
libcublas 11.6.5.2
libcudnn 8.2.4 | +| 1.12
1.11 | 11.4 | 8.2.4 (Linux)
8.2.2.26 (Windows) | libcudart 11.4.43
libcufft 10.5.2.100
libcurand 10.2.5.120
libcublasLt 11.6.5.2
libcublas 11.6.5.2
libcudnn 8.2.4 | +| 1.10 | 11.4 | 8.2.4 (Linux)
8.2.2.26 (Windows) | libcudart 11.4.43
libcufft 10.5.2.100
libcurand 10.2.5.120
libcublasLt 11.6.1.51
libcublas 11.6.1.51
libcudnn 8.2.4 | +| 1.9 | 11.4 | 8.2.4 (Linux)
8.2.2.26 (Windows) | libcudart 11.4.43
libcufft 10.5.2.100
libcurand 10.2.5.120
libcublasLt 11.6.1.51
libcublas 11.6.1.51
libcudnn 8.2.4 | +| 1.8 | 11.0.3 | 8.0.4 (Linux)
8.0.2.39 (Windows) | libcudart 11.0.221
libcufft 10.2.1.245
libcurand 10.2.1.245
libcublasLt 11.2.0.252
libcublas 11.2.0.252
libcudnn 8.0.4 | +| 1.7 | 11.0.3 | 8.0.4 (Linux)
8.0.2.39 (Windows) | libcudart 11.0.221
libcufft 10.2.1.245
libcurand 10.2.1.245
libcublasLt 11.2.0.252
libcublas 11.2.0.252
libcudnn 8.0.4 | + +### CUDA 10.x + +| ONNX Runtime | CUDA | cuDNN | Notes | +|--------------|--------|-------|-------------------------------------------------------------------------------------------------------------------| +| 1.5-1.6 | 10.2 | 8.0.3 | CUDA 11 can be built from source | +| 1.2-1.4 | 10.1 | 7.6.5 | Requires cublas10-10.2.1.243; cublas 10.1.x will not work | +| 1.0-1.1 | 10.0 | 7.6.4 | CUDA versions from 9.1 up to 10.1, and cuDNN versions from 7.1 up to 7.4 should also work with Visual Studio 2017 | For older versions, please reference the readme and build pages on the release branch.