mirror of
https://github.com/saymrwulf/onnxruntime.git
synced 2026-07-20 19:12:24 +00:00
[Doc] Update CUDA and cuDNN installation (#20035)
(1) Add CUDA and cuDNN installation section to ORT installation. (2) For build, TRT EP refers to CUDA EP for CUDA/cuDNN installation to avoid duplication. (3) Add comments about CUDA/cuDNN compatiblility
This commit is contained in:
Tianlei Wu
GitHub
parent
180cb20298
commit
a62200665e
3 changed files with 19 additions and 23 deletions
19
docs/build/eps.md
vendored
19
docs/build/eps.md
vendored
|
|
@ -45,11 +45,13 @@ The onnxruntime code will look for the provider shared libraries in the same loc
|
|||
### Prerequisites
|
||||
{: .no_toc }
|
||||
|
||||
* Install [CUDA](https://developer.nvidia.com/cuda-toolkit) and [cuDNN](https://developer.nvidia.com/cudnn) according to the [version compatibility matrix](../execution-providers/CUDA-ExecutionProvider.md#requirements).
|
||||
* The path to the CUDA installation must be provided via the CUDA_HOME environment variable, or the `--cuda_home` parameter.
|
||||
* The path to the cuDNN installation (include the `cuda` folder in the path) must be provided via the CUDNN_HOME environment variable, or `--cudnn_home` parameter. The cuDNN path should contain `bin`, `include` and `lib` directories.
|
||||
* The path to the cuDNN bin directory must be added to the PATH environment variable so that cudnn64_8.dll is found.
|
||||
|
||||
* Install [CUDA](https://developer.nvidia.com/cuda-toolkit) and [cuDNN](https://developer.nvidia.com/cudnn)
|
||||
* The CUDA execution provider for ONNX Runtime is built and tested with CUDA 11.8, 12.2 and cuDNN 8.9. Check [here](../execution-providers/CUDA-ExecutionProvider.md#requirements) for more version information.
|
||||
* The path to the CUDA installation must be provided via the CUDA_HOME environment variable, or the `--cuda_home` parameter. The installation directory should contain `bin`, `include` and `lib` sub-directories.
|
||||
* The path to the CUDA `bin` directory must be added to the PATH environment variable so that `nvcc` is found.
|
||||
* The path to the cuDNN installation must be provided via the CUDNN_HOME environment variable, or `--cudnn_home` parameter. In Windows, the installation directory should contain `bin`, `include` and `lib` sub-directories.
|
||||
* cuDNN 8.* requires ZLib. Follow the [cuDNN 8.9 installation guide](https://docs.nvidia.com/deeplearning/cudnn/archives/cudnn-890/install-guide/index.html) to install zlib in Linux or Windows.
|
||||
* In Windows, the path to the cuDNN bin directory must be added to the PATH environment variable so that cudnn64_8.dll is found.
|
||||
|
||||
### Build Instructions
|
||||
{: .no_toc }
|
||||
|
|
@ -106,12 +108,7 @@ See more information on the TensorRT Execution Provider [here](../execution-prov
|
|||
### Prerequisites
|
||||
{: .no_toc }
|
||||
|
||||
* Install [CUDA](https://developer.nvidia.com/cuda-toolkit) and [cuDNN](https://developer.nvidia.com/cudnn)
|
||||
* The TensorRT execution provider for ONNX Runtime is built and tested with CUDA 11.8, 12.2 and cuDNN 8.9. Check [here](https://onnxruntime.ai/docs/execution-providers/TensorRT-ExecutionProvider.html#requirements) for more version information.
|
||||
* The path to the CUDA installation must be provided via the CUDA_PATH environment variable, or the `--cuda_home` parameter. The CUDA path should contain `bin`, `include` and `lib` directories.
|
||||
* The path to the CUDA `bin` directory must be added to the PATH environment variable so that `nvcc` is found.
|
||||
* The path to the cuDNN installation (path to cudnn bin/include/lib) must be provided via the cuDNN_PATH environment variable, or `--cudnn_home` parameter.
|
||||
* On Windows, cuDNN requires [zlibwapi.dll](https://docs.nvidia.com/deeplearning/cudnn/installation/windows.html). Feel free to place this dll under `path_to_cudnn/bin`
|
||||
* Follow [instructions for CUDA execution provider](#cuda) to install CUDA and cuDNN, and setup environment variables.
|
||||
* Follow [instructions for installing TensorRT](https://docs.nvidia.com/deeplearning/tensorrt/install-guide/index.html)
|
||||
* The TensorRT execution provider for ONNX Runtime is built and tested with TensorRT 8.6.
|
||||
* The path to TensorRT installation must be provided via the `--tensorrt_home` parameter.
|
||||
|
|
|
|||
|
|
@ -33,15 +33,14 @@ Please reference table below for official GPU packages dependencies for the ONNX
|
|||
ONNX Runtime Training is aligned with PyTorch CUDA versions; refer to the Training tab
|
||||
on [onnxruntime.ai](https://onnxruntime.ai/) for supported versions.
|
||||
|
||||
Note: Because of CUDA Minor Version Compatibility, ONNX Runtime built with CUDA 11.8 should be compatible with any CUDA
|
||||
11.x version.
|
||||
Please
|
||||
reference [Nvidia CUDA Minor Version Compatibility](https://docs.nvidia.com/deploy/cuda-compatibility/#minor-version-compatibility).
|
||||
Note: Because of [Nvidia CUDA Minor Version Compatibility](https://docs.nvidia.com/deploy/cuda-compatibility/#minor-version-compatibility), ONNX Runtime built with CUDA 11.8 should be compatible with any CUDA 11.x version; ONNX Runtime built with CUDA 12.2 should be compatible with any CUDA 12.x version.
|
||||
|
||||
ONNX Runtime built with cuDNN 8.x are not compatible with cuDNN 9.x.
|
||||
|
||||
| ONNX Runtime | CUDA | cuDNN | Notes |
|
||||
|--------------------------|--------|-----------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| 1.17 | 12.2 | 8.9.2.26 (Linux)<br/>8.9.2.26 (Windows) | The default CUDA version for ORT 1.17 is CUDA 11.8. To install CUDA 12 package, please look at [Install ORT](../install).<br>Due to low demand on Java GPU package, only C++/C# Nuget and Python packages are released with CUDA 12.2 |
|
||||
| 1.15<br>1.16<br>1.17 | 11.8 | 8.2.4 (Linux)<br/>8.5.0.96 (Windows) | Tested with CUDA versions from 11.6 up to 11.8, and cuDNN from 8.2.4 up to 8.7.0 |
|
||||
| 1.15<br>1.16<br>1.17 | 11.8 | 8.2.4 (Linux)<br/>8.5.0.96 (Windows) | Tested with CUDA versions from 11.6 up to 11.8, and cuDNN from 8.2.4 up to 8.9.0 |
|
||||
| 1.14<br/>1.13.1<br/>1.13 | 11.6 | 8.2.4 (Linux)<br/>8.5.0.96 (Windows) | libcudart 11.4.43<br/>libcufft 10.5.2.100<br/>libcurand 10.2.5.120<br/>libcublasLt 11.6.5.2<br/>libcublas 11.6.5.2<br/>libcudnn 8.2.4 |
|
||||
| 1.12<br/>1.11 | 11.4 | 8.2.4 (Linux)<br/>8.2.2.26 (Windows) | libcudart 11.4.43<br/>libcufft 10.5.2.100<br/>libcurand 10.2.5.120<br/>libcublasLt 11.6.5.2<br/>libcublas 11.6.5.2<br/>libcudnn 8.2.4 |
|
||||
| 1.10 | 11.4 | 8.2.4 (Linux)<br/>8.2.2.26 (Windows) | libcudart 11.4.43<br/>libcufft 10.5.2.100<br/>libcurand 10.2.5.120<br/>libcublasLt 11.6.1.51<br/>libcublas 11.6.1.51<br/>libcudnn 8.2.4 |
|
||||
|
|
@ -54,10 +53,6 @@ reference [Nvidia CUDA Minor Version Compatibility](https://docs.nvidia.com/depl
|
|||
|
||||
For older versions, please reference the readme and build pages on the release branch.
|
||||
|
||||
For
|
||||
Windows, [Microsoft C and C++ (MSVC) runtime libraries](https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist)
|
||||
is also required.
|
||||
|
||||
## Build
|
||||
|
||||
For build instructions, please see the [BUILD page](../build/eps.md#cuda).
|
||||
|
|
|
|||
|
|
@ -31,6 +31,12 @@ under [Compatibility](../reference/compatibility).
|
|||
require [Visual C++ 2019 runtime](https://support.microsoft.com/en-us/help/2977003/the-latest-supported-visual-c-downloads).
|
||||
The latest version is recommended.
|
||||
|
||||
### CUDA and CuDNN
|
||||
For ONNX Runtime GPU package, it is required to install [CUDA](https://developer.nvidia.com/cuda-toolkit) and [cuDNN](https://developer.nvidia.com/cudnn). Check [CUDA execution provider requirements](../execution-providers/CUDA-ExecutionProvider.md#requirements) for compatible version of CUDA and cuDNN.
|
||||
* cuDNN 8.x requires ZLib. Follow the [cuDNN 8.9 installation guide](https://docs.nvidia.com/deeplearning/cudnn/archives/cudnn-890/install-guide/index.html) to install zlib in Linux or Windows. Note that the official gpu package does not support cuDNN 9.x.
|
||||
* The path of CUDA bin directory must be added to the PATH environment variable.
|
||||
* In Windows, the path of cuDNN bin directory must be added to the PATH environment variable.
|
||||
|
||||
## Python Installs
|
||||
|
||||
### Install ONNX Runtime (ORT)
|
||||
|
|
@ -42,15 +48,13 @@ pip install onnxruntime
|
|||
```
|
||||
|
||||
#### Install ONNX Runtime GPU (CUDA 11.x)
|
||||
|
||||
The default CUDA version for ORT is 11.8
|
||||
The default CUDA version for ORT is 11.8.
|
||||
|
||||
```bash
|
||||
pip install onnxruntime-gpu
|
||||
```
|
||||
|
||||
#### Install ONNX Runtime GPU (CUDA 12.x)
|
||||
|
||||
For Cuda 12.x, please use the following instructions to install from [ORT Azure Devops Feed](https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/onnxruntime-cuda-12/PyPI/onnxruntime-gpu/overview)
|
||||
|
||||
```bash
|
||||
|
|
|
|||
Loading…
Reference in a new issue