mirror of
https://github.com/saymrwulf/onnxruntime.git
synced 2026-07-14 18:12:05 +00:00
7e64928b06
### Description Added docs for ONNX 1.17 covering logging, tracing, and QNN EP Profiling ### Motivation and Context - ONNX Logging has not been documented - ONNX Tracing with Windows has barely been documented - ONNX 1.17 has new tracing and QNN EP Profiling PRs: #16259, #18201, #18882, #19397
2 KiB
2 KiB
| title | description | parent | grand_parent | has_children | nav_order |
|---|---|---|---|---|---|
| Configure CUDA for GPU with C# | Configure CUDA and cuDNN for ONNX Runtime with C# on Windows 11 | Inference with C# | Tutorials | false | 1 |
Configure CUDA and cuDNN for GPU with ONNX Runtime and C# on Windows 11
{: .no_toc }
Prerequisites
- Windows 11
- Visual Studio 2019 or 2022
Steps to Configure CUDA and cuDNN for ONNX Runtime with C# on Windows 11
-
Download and install the CUDA toolkit based on the supported version for the ONNX Runtime Version.
-
Download and install the cuDNN version based on the supported version for the ONNX Runtime Version.
See this table for supported versions:
| ONNX Runtime Version | CUDA Toolkit Version | cuDNN Version |
|---|---|---|
| 1.13 - 1.16 | 11.6 | 8.5.0.96 |
| 1.9 - 1.12 | 11.4 | 8.2.2.26 |
NOTE: Full table can be found here
-
Follow section 2. Installing cuDNN on Windows. NOTE: Skip step 5 in section 2.3 on updating Visual Studio settings, this is only for C++ projects.
-
Restart your computer and verify the installation by running the following command or in python with PyTorch:
nvcc --version
import torch
torch.cuda.is_available()
- Now you can enable GPU in the C# ONNX Runtime API with the following code:
// keep in mind almost all of the classes are disposable.
using var gpuSessionOptions = SessionOptions.MakeSessionOptionWithCudaProvider(0);
using var session = new InferenceSession(modelPath, gpuSessionOptions);