onnxruntime/docs/build/training.md

---
title: Build for training
parent: Build ONNX Runtime
description: Learn how to build ONNX Runtime for training from source for different hardware targets
nav_order: 2
redirect_from: /docs/how-to/build/training
---

# Build ONNX Runtime for training
{: .no_toc }

## Contents
{: .no_toc }

* TOC placeholder
{:toc}

## CPU

### Build Instructions
{: .no_toc }

To build ORT with training support add `--enable_training` build instruction.

All other build options are the same for inferencing as they are for training.

#### Windows

```
.\build.bat --config RelWithDebInfo --build_shared_lib --parallel --enable_training
```

The default Windows CMake Generator is Visual Studio 2017, but you can also use the newer Visual Studio 2019 by passing
`--cmake_generator "Visual Studio 16 2019"` to `.\build.bat`


#### Linux/macOS

```
./build.sh --config RelWithDebInfo --build_shared_lib --parallel --enable_training
```

## GPU / CUDA
### Prerequisites
{: .no_toc }

The default NVIDIA GPU build requires CUDA runtime libraries installed on the system:

* [CUDA](https://developer.nvidia.com/cuda-toolkit) 10.2
* [cuDNN](https://developer.nvidia.com/cudnn) 8.0
* [NCCL](https://developer.nvidia.com/nccl) 2.7
* [OpenMPI](https://www.open-mpi.org/) 4.0.4
  * See [install_openmpi.sh](https://github.com/microsoft/onnxruntime/blob/main/tools/ci_build/github/linux/docker/scripts/install_openmpi.sh)

These dependency versions should reflect what is in the [Dockerfiles](https://github.com/pytorch/ort/tree/main/docker).

### Build instructions
{: .no_toc }

1. Checkout this code repo with `git clone https://github.com/microsoft/onnxruntime`

2. Set the environment variables: *adjust the path for location your build machine*
    ```
    export CUDA_HOME=<location for CUDA libs> # e.g. /usr/local/cuda
    export CUDNN_HOME=<location for cuDNN libs> # e.g. /usr/local/cuda
    export CUDACXX=<location for NVCC> #e.g. /usr/local/cuda/bin/nvcc
    export PATH=<location for openmpi/bin/>:$PATH
    export LD_LIBRARY_PATH=<location for openmpi/lib/>:$LD_LIBRARY_PATH
    export MPI_CXX_INCLUDE_PATH=<location for openmpi/include/>
    ```

3. Create the ONNX Runtime wheel

   * Change to the ONNX Runtime repo base folder: `cd onnxruntime`
   * Run `./build.sh --enable_training --use_cuda --config=RelWithDebInfo --build_wheel`

    This produces the .whl file in `./build/Linux/RelWithDebInfo/dist` for ONNX Runtime Training.

## GPU / ROCm
### Prerequisites
{: .no_toc }

The default AMD GPU build requires ROCm software toolkit installed on the system:

* [ROCm](https://docs.amd.com/bundle/ROCm-Installation-Guide-v5.4/page/How_to_Install_ROCm.html#_How_to_Install) 5.4
* [OpenMPI](https://www.open-mpi.org/) 4.0.4
  * See [install_openmpi.sh](https://github.com/microsoft/onnxruntime/blob/main/tools/ci_build/github/linux/docker/scripts/install_openmpi.sh)

### Build instructions
{: .no_toc }

1. Checkout this code repo with `git clone https://github.com/microsoft/onnxruntime`

2. Create the ONNX Runtime wheel

   * Change to the ONNX Runtime repo base folder: `cd onnxruntime`
   * Run `./build.sh --config Release --enable_training --build_wheel --parallel --skip_tests --use_rocm --rocm_home /opt/rocm --nccl_home /opt/rocm --mpi_home <location for openmpi>`

    This produces the .whl file in `./build/Linux/Release/dist` for ONNX Runtime Training.

## DNNL and MKLML

### Build Instructions
{: .no_toc }
#### Linux

`./build.sh --enable_training --use_dnnl`

#### Windows

`.\build.bat --enable_training --use_dnnl`

Add `--build_wheel` to build the ONNX Runtime wheel.

This will produce a .whl file in `build/Linux/RelWithDebInfo/dist` for ONNX Runtime Training.