saymrwulf/onnxruntime
1
0
Fork 0
mirror of https://github.com/saymrwulf/onnxruntime.git synced 2026-07-18 18:52:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

How to setup the cuda env for windows (#20444)

Browse source
This commit is contained in:
Baiju Meswani 2024-04-24 16:35:47 -07:00 committed by GitHub
parent faf0809e25
commit 5e549a5c24
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 24 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

24
docs/genai/howto/setup-cuda-env.md Normal file
View file

@ -0,0 +1,24 @@
---
title: Setup CUDA env
description: Instructions to setup the CUDA environtment to run onnxruntime-genai-cuda
has_children: false
parent: How to
grand_parent: Generative AI (Preview)
nav_order: 4
---
# Setup the CUDA Environment
{: .no_toc }
* TOC placeholder
{:toc}
## Install the CUDA Toolkit
On a CUDA capable machine, install the CUDA toolkit. onnxruntime-genai-cuda is built and packaged with CUDA-11.8.
The CUDA toolkit can be downloaded from the [CUDA Toolkit Archive](https://developer.nvidia.com/cuda-toolkit-archive).
## Special Instructions on Windows
After CUDA toolkit installation completed on windows, ensure that the `CUDA_PATH` system environment variable has been set to the path where the toolkit was installed. This variable will be used when importing the onnxruntime_genai python module on windows. Unset or incorrectly set `CUDA_PATH` variable may lead to a `DLL load failed while importing onnxruntime_genai`.