mirror of
https://github.com/saymrwulf/onnxruntime.git
synced 2026-07-14 18:12:05 +00:00
Improve generate() API build instructions (#20181)
Staged here: https://natke.github.io/onnxruntime/docs/genai/howto/build-from-source.html
This commit is contained in:
Nat Kershaw (MSFT)
GitHub
parent
e31928e7ff
commit
4a0a4ce205
1 changed files with 85 additions and 70 deletions
|
|
@ -17,105 +17,120 @@ nav_order: 2
|
|||
|
||||
`cmake`
|
||||
|
||||
## Build steps
|
||||
## Clone the onnxruntime-genai repo
|
||||
|
||||
1. Clone this repo
|
||||
```bash
|
||||
git clone https://github.com/microsoft/onnxruntime-genai
|
||||
cd onnxruntime-genai
|
||||
```
|
||||
|
||||
```bash
|
||||
git clone https://github.com/microsoft/onnxruntime-genai
|
||||
cd onnxruntime-genai
|
||||
```
|
||||
## Install ONNX Runtime
|
||||
|
||||
2. Install ONNX Runtime
|
||||
By default, the onnxruntime-genai build expects to find the ONNX Runtime include and binaries in a folder called `ort` in the root directory of onnxruntime-genai. You can put the ONNX Runtime files in a different location and specify this location to the onnxruntime-genai build. These instructions use `ORT_HOME` as the location.
|
||||
|
||||
By default, the onnxruntime-genai build expects to find the ONNX Runtime include and binaries in a folder called `ort` in the root directory of onnxruntime-genai. You can put the ONNX Runtime files in a different location and specify this location to the onnxruntime-genai build. These instructions use ORT_HOME as the location.
|
||||
### Option 1: Install from release
|
||||
|
||||
* Install from release
|
||||
These instructions are for the Linux GPU build of ONNX Runtime. Replace `linux-gpu` with your target of choice.
|
||||
|
||||
These instructions are for the Linux GPU build of ONNX Runtime. Replace the location with the operating system and target of choice.
|
||||
```bash
|
||||
cd <ORT_HOME>
|
||||
curl -L https://github.com/microsoft/onnxruntime/releases/download/v1.17.0/onnxruntime-linux-x64-gpu-1.17.1.tgz
|
||||
tar xvzf onnxruntime-linux-x64-gpu-1.17.1.tgz
|
||||
mv onnxruntime-linux-x64-gpu-1.17.1/include .
|
||||
mv onnxruntime-linux-x64-gpu-1.17.1/lib .
|
||||
```
|
||||
|
||||
```bash
|
||||
cd $ORT_HOME
|
||||
wget https://github.com/microsoft/onnxruntime/releases/download/v1.17.0/onnxruntime-linux-x64-gpu-1.17.0.tgz
|
||||
tar xvzf onnxruntime-linux-x64-gpu-1.17.0.tgz
|
||||
mv onnxruntime-linux-x64-gpu-1.17.0/include .
|
||||
mv onnxruntime-linux-x64-gpu-1.17.0/lib .
|
||||
```
|
||||
### Option 2: Install from nightly
|
||||
|
||||
* Install from nightly
|
||||
|
||||
Download the nightly nuget package `Microsoft.ML.OnnxRuntime` from: https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/ORT-Nightly.
|
||||
Download the nightly nuget package `Microsoft.ML.OnnxRuntime` from: https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/ORT-Nightly.
|
||||
|
||||
Extract the nuget package.
|
||||
Extract the nuget package.
|
||||
|
||||
```bash
|
||||
tar xvf Microsoft.ML.OnnxRuntime.1.18.0-dev-20240322-0323-ca825cb6e6.nupkg
|
||||
```
|
||||
```bash
|
||||
tar xvf Microsoft.ML.OnnxRuntime.1.18.0-dev-20240322-0323-ca825cb6e6.nupkg
|
||||
```
|
||||
|
||||
Copy the include and lib files into $ORT_HOME.
|
||||
Copy the include and lib files into `ORT_HOME`.
|
||||
|
||||
On Windows
|
||||
On Windows
|
||||
|
||||
Example is given for `win-x64`. Change this to your architecture if different.
|
||||
Example is given for `win-x64`. Change this to your architecture if different.
|
||||
|
||||
```cmd
|
||||
copy build\native\include\onnxruntime_c_api.h $ORT_HOME\include
|
||||
copy runtimes\win-x64\native\*.dll $ORT_HOME\lib
|
||||
```
|
||||
```cmd
|
||||
copy build\native\include\onnxruntime_c_api.h <ORT_HOME>\include
|
||||
copy runtimes\win-x64\native\*.dll <ORT_HOME>\lib
|
||||
```
|
||||
|
||||
On Linux
|
||||
On Linux
|
||||
|
||||
```cmd
|
||||
cp build/native/include/onnxruntime_c_api.h $ORT_HOME/include
|
||||
cp build/linux-x64/native/libonnxruntime*.so* $ORT_HOME/lib
|
||||
```
|
||||
```cmd
|
||||
cp build/native/include/onnxruntime_c_api.h <ORT_HOME>/include
|
||||
cp build/linux-x64/native/libonnxruntime*.so* <ORT_HOME>/lib
|
||||
```
|
||||
|
||||
* Or build from source
|
||||
### Option 3: Build from source
|
||||
|
||||
```
|
||||
git clone https://github.com/microsoft/onnxruntime.git
|
||||
cd onnxruntime
|
||||
```
|
||||
```
|
||||
git clone https://github.com/microsoft/onnxruntime.git
|
||||
cd onnxruntime
|
||||
```
|
||||
|
||||
Create include and lib folders in the ORT_HOME directory
|
||||
Create include and lib folders in the `ORT_HOME` directory
|
||||
|
||||
```bash
|
||||
mkdir $ORT_HOME/include
|
||||
mkdir $ORT_HOME/lib
|
||||
```
|
||||
```bash
|
||||
mkdir <ORT HOME>/include
|
||||
mkdir <ORT_HOME>/lib
|
||||
```
|
||||
|
||||
Build from source and copy the include and libraries into ORT_HOME
|
||||
Build from source and copy the include and libraries into `ORT_HOME`
|
||||
|
||||
On Windows
|
||||
On Windows
|
||||
|
||||
```cmd
|
||||
build.bat --build_shared_lib --skip_tests --parallel [--use_cuda]
|
||||
copy include\onnxruntime\core\session\onnxruntime_c_api.h $ORT_HOME\include
|
||||
copy build\Windows\Debug\Debug\*.dll $ORT_HOME\lib
|
||||
```
|
||||
```cmd
|
||||
build.bat --build_shared_lib --skip_tests --parallel [--use_cuda]
|
||||
copy include\onnxruntime\core\session\onnxruntime_c_api.h <ORT_HOME>\include
|
||||
copy build\Windows\Debug\Debug\*.dll <ORT_HOME>\lib
|
||||
```
|
||||
|
||||
On Linux
|
||||
On Linux
|
||||
|
||||
```cmd
|
||||
./build.sh --build_shared_lib --skip_tests --parallel [--use_cuda]
|
||||
cp include/onnxruntime/core/session/onnxruntime_c_api.h $ORT_HOME/include
|
||||
cp build/Linux/RelWithDebInfo/libonnxruntime*.so* $ORT_HOME/lib
|
||||
```
|
||||
```cmd
|
||||
./build.sh --build_shared_lib --skip_tests --parallel [--use_cuda]
|
||||
cp include/onnxruntime/core/session/onnxruntime_c_api.h <ORT_HOME>/include
|
||||
cp build/Linux/RelWithDebInfo/libonnxruntime*.so* <ORT_HOME>/lib
|
||||
```
|
||||
|
||||
3. Build onnxruntime-genai
|
||||
## Build onnxruntime-genai
|
||||
|
||||
If you are building for CUDA, add the cuda_home argument.
|
||||
### Build for CPU
|
||||
|
||||
```bash
|
||||
cd ..
|
||||
python build.py [--cuda_home <path_to_cuda_home>]
|
||||
```
|
||||
```bash
|
||||
cd ..
|
||||
python build.py [--ort_home <ORT_HOME>]
|
||||
```
|
||||
|
||||
### Build for CUDA
|
||||
|
||||
These instructions assume you already have CUDA installed.
|
||||
|
||||
```bash
|
||||
cd ..
|
||||
python build.py --cuda_home <path to cuda home> [--ort_home <ORT_HOME>]
|
||||
```
|
||||
|
||||
4. Install Python wheel
|
||||
## Install the library into your application
|
||||
|
||||
```bash
|
||||
cd build/wheel
|
||||
pip install *.whl
|
||||
```
|
||||
### Install Python wheel
|
||||
|
||||
```bash
|
||||
cd build/wheel
|
||||
pip install *.whl
|
||||
```
|
||||
|
||||
### Install Nuget package
|
||||
|
||||
_Coming soon_
|
||||
|
||||
### Install C/C++ header file and library
|
||||
|
||||
_Coming soon_
|
||||
Loading…
Reference in a new issue