saymrwulf/onnxruntime
1
0
Fork 0
mirror of https://github.com/saymrwulf/onnxruntime.git synced 2026-07-17 18:40:28 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

[Doc] Update GPU package installation (#22204)

Browse source 
### Description
* Add CUDA/CuDNN requirements for 1.19.x
* Update installation guide for cuda 11 package and nightly.
* Remove a sentence for ROCm: `Alternatively, each major ORT release has
a corresponding C/C++ ROCm package, found
[here](https://github.com/microsoft/onnxruntime/releases/).` since
onnxruntime_rocm package is not available in recent releases.

### Motivation and Context
Doc is not up to date.
This commit is contained in:
Tianlei Wu 2024-09-24 13:57:20 -07:00 committed by GitHub
parent 85c3f0fe0b
commit bc1bce956a
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
4 changed files with 27 additions and 23 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

8
docs/execution-providers/CUDA-ExecutionProvider.md
View file

@ -35,14 +35,13 @@ Because of [Nvidia CUDA Minor Version Compatibility](https://docs.nvidia.com/dep
ONNX Runtime built with cuDNN 8.x is not compatible with cuDNN 9.x, and vice versa. You can choose the package based on CUDA and cuDNN major versions that match your runtime environment (For example, PyTorch 2.3 uses cuDNN 8.x, while PyTorch 2.4 or later used cuDNN 9.x).
Note: starting ORT 1.19, **CUDA 12.x** becomes default version when distributing ONNX Runtime GPU packages.
Note: starting ORT 1.19, **CUDA 12.x** becomes default version when distributing ONNX Runtime GPU packages in pypi.
### CUDA 12.x
To install CUDA 12 package, please look at [Install ORT](../install).
| ONNX Runtime | CUDA | cuDNN | Notes |
|---------------|--------|-------|----------------------------------------------------------------------|
| 1.19.x | 12.x | 9.x | Avaiable in pypi. Compatible with PyTorch >= 2.4.0 for cuda 12.x. |
| 1.18.1 | 12.x | 9.x | cuDNN 9 is required. No Java package. |
| 1.18.0 | 12.x | 8.x | Java package is added. |
| 1.17.x | 12.x | 8.x | Only C++/C# Nuget and Python packages are released. No Java package. |
@ -51,7 +50,8 @@ To install CUDA 12 package, please look at [Install ORT](../install).
| ONNX Runtime | CUDA | cuDNN | Notes |
|----------------------|--------|-----------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------|
| 1.18.x | 11.8 | 8.x | |
| 1.19.x | 11.8 | 8.x | Not available in pypi. See [Install ORT](../install) for detail. Compatible with PyTorch <= 2.3.1 for CUDA 11.8. |
| 1.18.x | 11.8 | 8.x | Available in pypi |
| 1.17<br>1.16<br>1.15 | 11.8 | 8.2.4 (Linux)<br/>8.5.0.96 (Windows) | Tested with CUDA versions from 11.6 up to 11.8, and cuDNN from 8.2 up to 8.9 |
| 1.14<br>1.13 | 11.6 | 8.2.4 (Linux)<br/>8.5.0.96 (Windows) | libcudart 11.4.43<br/>libcufft 10.5.2.100<br/>libcurand 10.2.5.120<br/>libcublasLt 11.6.5.2<br/>libcublas 11.6.5.2<br/>libcudnn 8.2.4 |
| 1.12<br>1.11 | 11.4 | 8.2.4 (Linux)<br/>8.2.2.26 (Windows) | libcudart 11.4.43<br/>libcufft 10.5.2.100<br/>libcurand 10.2.5.120<br/>libcublasLt 11.6.5.2<br/>libcublas 11.6.5.2<br/>libcudnn 8.2.4 |

6
docs/execution-providers/TensorRT-ExecutionProvider.md
View file

@ -31,9 +31,9 @@ Note: starting ORT 1.19, **CUDA 12** becomes default version when distributing O
| ONNX Runtime | TensorRT | CUDA |
| :----------- | :------- | :------------- |
| 1.19-main | 10.2 | **12.2**, 11.8 |
| 1.18 | 10.0 | 11.8, 12.2 |
| 1.17 | 8.6 | 11.8, 12.2 |
| 1.19-main | 10.2 | **12.x**, 11.8 |
| 1.18 | 10.0 | 11.8, 12.x |
| 1.17 | 8.6 | 11.8, 12.x |
| 1.16 | 8.6 | 11.8 |
| 1.15 | 8.6 | 11.8 |
| 1.14 | 8.5 | 11.6 |

10
docs/get-started/with-python.md
View file

@ -30,7 +30,7 @@ pip install onnxruntime
### Install ONNX Runtime GPU (CUDA 12.x)
The default CUDA version for ORT is 12.2.
The default CUDA version for ORT is 12.x.
```bash
pip install onnxruntime-gpu
@ -260,8 +260,8 @@ If using pip, run `pip install --upgrade pip` prior to downloading.
|[onnxruntime](https://pypi.org/project/onnxruntime)|CPU (Release)| Windows (x64), Linux (x64, ARM64), Mac (X64), |
|[ort-nightly](https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/ORT-Nightly/PyPI/ort-nightly)|CPU (Dev) | Same as above |
|[onnxruntime-gpu](https://pypi.org/project/onnxruntime-gpu)|GPU (Release)| Windows (x64), Linux (x64, ARM64) |
|[ort-nightly-gpu for CUDA 11.*](https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/ORT-Nightly/PyPI/ort-nightly-gpu) |GPU (Dev) | Windows (x64), Linux (x64, ARM64) |
|[ort-nightly-gpu for CUDA 12.*](https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/ort-cuda-12-nightly/PyPI/ort-nightly-gpu) |GPU (Dev) | Windows (x64), Linux (x64, ARM64) |
|[ort-nightly-gpu for CUDA 11.*](https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/ort-cuda-11-nightly/PyPI/ort-nightly-gpu) |GPU (Dev) | Windows (x64), Linux (x64, ARM64) |
|[ort-nightly-gpu for CUDA 12.*](https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/ORT-Nightly/PyPI/ort-nightly-gpu) |GPU (Dev) | Windows (x64), Linux (x64, ARM64) |
Before installing nightly package, you will need install dependencies first.
```
@ -270,12 +270,12 @@ python -m pip install coloredlogs flatbuffers numpy packaging protobuf sympy
Example to install ort-nightly-gpu for CUDA 11.*:
```
python -m pip install ort-nightly-gpu --index-url=https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/
python -m pip install ort-nightly-gpu --index-url=https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ort-cuda-11-nightly/pypi/simple/
```
Example to install ort-nightly-gpu for CUDA 12.*:
```
python -m pip install ort-nightly-gpu --index-url=https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ort-cuda-12-nightly/pypi/simple/
python -m pip install ort-nightly-gpu --index-url=https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/
```
For Python compiler version notes, see [this page](https://github.com/microsoft/onnxruntime/tree/main/docs/Python_Dev_Notes.md)

26
docs/install/index.md
View file

@ -46,25 +46,29 @@ For ONNX Runtime GPU package, it is required to install [CUDA](https://developer
pip install onnxruntime
```
#### Install ONNX Runtime GPU (CUDA 11.x)
The default CUDA version for ORT is 11.8.
#### Install ONNX Runtime GPU (CUDA 12.x)
The default CUDA version for ORT in pypi is 12.x since 1.19.0.
```bash
pip install onnxruntime-gpu
```
#### Install ONNX Runtime GPU (CUDA 12.x)
For Cuda 12.x, please use the following instructions to install from [ORT Azure Devops Feed](https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/onnxruntime-cuda-12/PyPI/onnxruntime-gpu/overview)
#### Install ONNX Runtime GPU (CUDA 11.x)
For Cuda 11.x, please use the following instructions to install from [ORT Azure Devops Feed](https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/onnxruntime-cuda-11/PyPI/onnxruntime-gpu/overview) for 1.19.2 or later.
```bash
pip install onnxruntime-gpu --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-12/pypi/simple/
pip install onnxruntime-gpu --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-11/pypi/simple/
```
For older version <= 1.18.1, you can download from pypi with explicit version like the following:
```bash
pip install onnxruntime-gpu==1.18.1
```
#### Install ONNX Runtime GPU (ROCm)
For ROCm, please follow instructions to install it at the [AMD ROCm install docs](https://rocm.docs.amd.com/projects/install-on-linux/en/docs-6.0.0/). The ROCm execution provider for ONNX Runtime is built and tested with ROCm 6.0.0
To build from source on Linux, follow the instructions [here](https://onnxruntime.ai/docs/build/eps.html#amd-rocm). Alternatively, each major ORT release has a corresponding C/C++ ROCm package, found [here](https://github.com/microsoft/onnxruntime/releases/).
For ROCm, please follow instructions to install it at the [AMD ROCm install docs](https://rocm.docs.amd.com/projects/install-on-linux/en/docs-6.0.0/). The ROCm execution provider for ONNX Runtime is built and tested with ROCm 6.0.0.
To build from source on Linux, follow the instructions [here](https://onnxruntime.ai/docs/build/eps.html#amd-rocm).
### Install ONNX to export the model
@ -96,7 +100,7 @@ dotnet add package Microsoft.ML.OnnxRuntime
#### Install ONNX Runtime GPU (CUDA 12.x)
The default CUDA version for ORT is 12.2
The default CUDA version for ORT is 12.x
```bash
# GPU
@ -405,8 +409,8 @@ below:
|--------------|---------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------|
| Python | If using pip, run `pip install --upgrade pip` prior to downloading. | | |
| | CPU: [**onnxruntime**](https://pypi.org/project/onnxruntime) | [ort-nightly (dev)](https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/ORT-Nightly/PyPI/ort-nightly/overview) | |
| | GPU (CUDA/TensorRT) for CUDA 11.x: [**onnxruntime-gpu**](https://pypi.org/project/onnxruntime-gpu) | [ort-nightly-gpu (dev)](https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/ORT-Nightly/PyPI/ort-nightly-gpu/overview/) | [View](../execution-providers/CUDA-ExecutionProvider.md#requirements) |
| | GPU (CUDA/TensorRT) for CUDA 12.x: [**onnxruntime-gpu**](https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/onnxruntime-cuda-12/PyPI/onnxruntime-gpu/overview/) | [ort-nightly-gpu (dev)](https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/ort-cuda-12-nightly/PyPI/ort-nightly-gpu/overview/) | [View](../execution-providers/CUDA-ExecutionProvider.md#requirements) |
| | GPU (CUDA/TensorRT) for CUDA 12.x: [**onnxruntime-gpu**](https://pypi.org/project/onnxruntime-gpu) | [ort-nightly-gpu (dev)](https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/ORT-Nightly/PyPI/ort-nightly-gpu/overview/) | [View](../execution-providers/CUDA-ExecutionProvider.md#requirements) |
| | GPU (CUDA/TensorRT) for CUDA 11.x: [**onnxruntime-gpu**](https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/onnxruntime-cuda-11/PyPI/onnxruntime-gpu/overview/) | [ort-nightly-gpu (dev)](https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/ort-cuda-11-nightly/PyPI/ort-nightly-gpu/overview/) | [View](../execution-providers/CUDA-ExecutionProvider.md#requirements) |
| | GPU (DirectML): [**onnxruntime-directml**](https://pypi.org/project/onnxruntime-directml/) | [ort-nightly-directml (dev)](https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/ORT-Nightly/PyPI/ort-nightly-directml/overview/) | [View](../execution-providers/DirectML-ExecutionProvider.md#requirements) |
| | OpenVINO: [**intel/onnxruntime**](https://github.com/intel/onnxruntime/releases/latest) - *Intel managed* | | [View](../build/eps.md#openvino) |
| | TensorRT (Jetson): [**Jetson Zoo**](https://elinux.org/Jetson_Zoo#ONNX_Runtime) - *NVIDIA managed* | | |