mirror of
https://github.com/saymrwulf/onnxruntime.git
synced 2026-07-29 20:14:01 +00:00
Training related changes to the onnxruntime website (#15900)
Changes include: - Added a new page for `On-Device Training` overview: [Preview](https://baijumeswani.github.io/onnxruntime/docs/get-started/on-device-training.html) - Added a new section for `On-Device Training` installation: [Preview](https://baijumeswani.github.io/onnxruntime/docs/install/#install-for-on-device-training) - Added a new section for `On-Device Training` build from source: [Preview](https://baijumeswani.github.io/onnxruntime/docs/build/training.html#build-for-on-device-training) - Updated Large Model Training overview, installation, build pages to reflect what is currently accurate. Website preview: https://baijumeswani.github.io/onnxruntime/ Pending website work: - Update links for released packages for training. - Add tutorial for on-device training - Add links to the blog posts that detail on device training.
This commit is contained in:
Baiju Meswani
GitHub
parent
e2dda61278
commit
25125b5b91
14 changed files with 273 additions and 110 deletions
|
|
@ -8,7 +8,7 @@ redirect_from: /docs/reference/api
|
|||
{: .no_toc }
|
||||
|
||||
|:----------------------------------------------------------------------------------|
|
||||
| <span class="fs-5"> [Python API Docs](python/api_summary.html){: .btn target="_blank"} </span> |
|
||||
| <span class="fs-5"> [Python API Docs](python/index.html){: .btn target="_blank"} </span> |
|
||||
| <span class="fs-5"> [Java API Docs](java/index.html){: .btn target="_blank"} </span> |
|
||||
| <span class="fs-5"> [C# API Docs](csharp/api){: .btn target="_blank"} </span>|
|
||||
| <span class="fs-5"> [C/C++ API Docs](c){: .btn target="_blank"} </span>|
|
||||
|
|
|
|||
0
docs/api/python/index.html
Normal file
0
docs/api/python/index.html
Normal file
138
docs/build/training.md
vendored
138
docs/build/training.md
vendored
|
|
@ -1,12 +1,67 @@
|
|||
---
|
||||
title: Build for training
|
||||
parent: Build ONNX Runtime
|
||||
description: Learn how to build ONNX Runtime for training from source for different hardware targets
|
||||
description: Learn how to build ONNX Runtime for training from source for different scenarios and hardware targets
|
||||
nav_order: 2
|
||||
redirect_from: /docs/how-to/build/training
|
||||
---
|
||||
|
||||
# Build ONNX Runtime for training
|
||||
# Build for On-Device Training
|
||||
{: .no_toc }
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Python 3.x
|
||||
- CMake
|
||||
|
||||
## Build Instructions for the Training Phase
|
||||
|
||||
1. Clone the repository
|
||||
|
||||
```bash
|
||||
git clone --recursive https://github.com/Microsoft/onnxruntime.git
|
||||
cd onnxruntime
|
||||
```
|
||||
|
||||
2. Build ONNX Runtime for `On-Device Training`
|
||||
|
||||
a. For Windows
|
||||
|
||||
```powershell
|
||||
.\build.bat --config RelWithDebInfo --cmake_generator "Visual Studio 17 2022" --build_shared_lib --parallel --enable_training_apis
|
||||
```
|
||||
|
||||
b. For Linux
|
||||
|
||||
```bash
|
||||
./build.sh --config RelWithDebInfo --build_shared_lib --parallel --enable_training_apis
|
||||
```
|
||||
|
||||
c. For Android
|
||||
|
||||
Refer to the [Android build instructions](./android.md) and add the ```--enable_training_apis``` build flag.
|
||||
|
||||
> **Note**
|
||||
>
|
||||
> - To build the C# bindings, add the ```--build_nuget``` flag to the build command above.
|
||||
>
|
||||
> - To build the Python wheel:
|
||||
> - add the ```--build_wheel``` flag to the build command above.
|
||||
> - install the wheel using ```python -m pip install build/Linux/RelWithDebInfo/dist/*.whl```
|
||||
>
|
||||
> - The ```config``` flag can be one of ```Debug```, ```RelWithDebInfo```, ```Release```, ```MinSizeRel```. Use the one that suits your use case.
|
||||
>
|
||||
> - The ```--enable_training_apis``` flag can be used in conjunction with the ```--minimal_build``` flag.
|
||||
>
|
||||
> - The offline phase of generating the training artifacts can only be done with Python (using the ```--build_wheel``` flag).
|
||||
>
|
||||
> - The build commands above only build for the cpu execution provider. To build for cuda execution provider, add these flags
|
||||
> - ```--use_cuda```
|
||||
> - ```--cuda_home {directory to your cuda home, for example /usr/local/cuda/}```
|
||||
> - ```--cudnn_home {directory to your cuda home, for example /usr/local/cuda/}```
|
||||
> - ```--cuda_version={version for example 11.8}```
|
||||
|
||||
# Build for Large Model Training
|
||||
{: .no_toc }
|
||||
|
||||
## Contents
|
||||
|
|
@ -15,26 +70,7 @@ redirect_from: /docs/how-to/build/training
|
|||
* TOC placeholder
|
||||
{:toc}
|
||||
|
||||
## CPU
|
||||
|
||||
### Build Instructions
|
||||
{: .no_toc }
|
||||
|
||||
To build ORT with training support add `--enable_training` build instruction.
|
||||
|
||||
All other build options are the same for inferencing as they are for training.
|
||||
|
||||
#### Windows
|
||||
|
||||
```
|
||||
.\build.bat --config RelWithDebInfo --build_shared_lib --parallel --enable_training
|
||||
```
|
||||
|
||||
The default Windows CMake Generator is Visual Studio 2017, but you can also use the newer Visual Studio 2019 by passing
|
||||
`--cmake_generator "Visual Studio 16 2019"` to `.\build.bat`
|
||||
|
||||
|
||||
#### Linux/macOS
|
||||
#### Linux
|
||||
|
||||
```
|
||||
./build.sh --config RelWithDebInfo --build_shared_lib --parallel --enable_training
|
||||
|
|
@ -46,35 +82,39 @@ The default Windows CMake Generator is Visual Studio 2017, but you can also use
|
|||
|
||||
The default NVIDIA GPU build requires CUDA runtime libraries installed on the system:
|
||||
|
||||
* [CUDA](https://developer.nvidia.com/cuda-toolkit) 10.2
|
||||
* [cuDNN](https://developer.nvidia.com/cudnn) 8.0
|
||||
* [NCCL](https://developer.nvidia.com/nccl) 2.7
|
||||
* [OpenMPI](https://www.open-mpi.org/) 4.0.4
|
||||
* See [install_openmpi.sh](https://github.com/microsoft/onnxruntime/blob/main/tools/ci_build/github/linux/docker/scripts/install_openmpi.sh)
|
||||
|
||||
These dependency versions should reflect what is in the [Dockerfiles](https://github.com/pytorch/ort/tree/main/torch_ort/docker).
|
||||
* [CUDA](https://developer.nvidia.com/cuda-toolkit)
|
||||
* [cuDNN](https://developer.nvidia.com/cudnn)
|
||||
|
||||
### Build instructions
|
||||
{: .no_toc }
|
||||
|
||||
1. Checkout this code repo with `git clone https://github.com/microsoft/onnxruntime`
|
||||
1. Checkout this code repo with
|
||||
|
||||
2. Set the environment variables: *adjust the path for location your build machine*
|
||||
```bash
|
||||
git clone https://github.com/microsoft/onnxruntime
|
||||
cd onnxruntime
|
||||
```
|
||||
|
||||
2. Set the environment variables: *adjust the paths for locations on your build machine*
|
||||
```bash
|
||||
export CUDA_HOME=<location for CUDA libs> # e.g. /usr/local/cuda
|
||||
export CUDNN_HOME=<location for cuDNN libs> # e.g. /usr/local/cuda
|
||||
export CUDACXX=<location for NVCC> #e.g. /usr/local/cuda/bin/nvcc
|
||||
export PATH=<location for openmpi/bin/>:$PATH
|
||||
export LD_LIBRARY_PATH=<location for openmpi/lib/>:$LD_LIBRARY_PATH
|
||||
export MPI_CXX_INCLUDE_PATH=<location for openmpi/include/>
|
||||
```
|
||||
|
||||
3. Create the ONNX Runtime wheel
|
||||
3. Create the ONNX Runtime Python wheel
|
||||
|
||||
* Change to the ONNX Runtime repo base folder: `cd onnxruntime`
|
||||
* Run `./build.sh --enable_training --use_cuda --config=RelWithDebInfo --build_wheel`
|
||||
```bash
|
||||
./build.sh --config=RelWithDebInfo --enable_training --build_wheel --use_cuda --cuda_home {location of cuda libs eg. /usr/local/cuda/} --cudnn_home {location of cudnn libs eg./usr/local/cuda/} --cuda_version={version for eg. 11.8}
|
||||
```
|
||||
|
||||
This produces the .whl file in `./build/Linux/RelWithDebInfo/dist` for ONNX Runtime Training.
|
||||
4. Install the .whl file in `./build/Linux/RelWithDebInfo/dist` for ONNX Runtime Training.
|
||||
|
||||
```bash
|
||||
python -m pip install build/Linux/RelWithDebInfo/dist/*.whl
|
||||
```
|
||||
|
||||
That's it! Once the build is complete, you should be able to use the ONNX Runtime libraries and executables in your projects. Note that these steps are general and may need to be adjusted based on your specific environment and requirements. For more information, you can ask for help on the [ONNX Runtime GitHub community](https://github.com/microsoft/onnxruntime/discussions/new?category=q-a).
|
||||
|
||||
## GPU / ROCm
|
||||
### Prerequisites
|
||||
|
|
@ -83,20 +123,28 @@ These dependency versions should reflect what is in the [Dockerfiles](https://gi
|
|||
The default AMD GPU build requires ROCm software toolkit installed on the system:
|
||||
|
||||
* [ROCm](https://docs.amd.com/bundle/ROCm-Installation-Guide-v5.4/page/How_to_Install_ROCm.html#_How_to_Install) 5.4
|
||||
* [OpenMPI](https://www.open-mpi.org/) 4.0.4
|
||||
* See [install_openmpi.sh](https://github.com/microsoft/onnxruntime/blob/main/tools/ci_build/github/linux/docker/scripts/install_openmpi.sh)
|
||||
|
||||
### Build instructions
|
||||
{: .no_toc }
|
||||
|
||||
1. Checkout this code repo with `git clone https://github.com/microsoft/onnxruntime`
|
||||
1. Checkout this code repo with
|
||||
|
||||
2. Create the ONNX Runtime wheel
|
||||
```bash
|
||||
git clone https://github.com/microsoft/onnxruntime
|
||||
cd onnxruntime
|
||||
```
|
||||
|
||||
* Change to the ONNX Runtime repo base folder: `cd onnxruntime`
|
||||
* Run `./build.sh --config Release --enable_training --build_wheel --parallel --skip_tests --use_rocm --rocm_home /opt/rocm --nccl_home /opt/rocm --mpi_home <location for openmpi>`
|
||||
2. Create the ONNX Runtime Python wheel
|
||||
|
||||
This produces the .whl file in `./build/Linux/Release/dist` for ONNX Runtime Training.
|
||||
```bash
|
||||
./build.sh --config Release --enable_training --build_wheel --parallel --skip_tests --use_rocm --rocm_home /opt/rocm
|
||||
```
|
||||
|
||||
3. Install the .whl file in `./build/Linux/RelWithDebInfo/dist` for ONNX Runtime Training.
|
||||
|
||||
```bash
|
||||
python -m pip install build/Linux/RelWithDebInfo/dist/*.whl
|
||||
```
|
||||
|
||||
## DNNL and MKLML
|
||||
|
||||
|
|
|
|||
48
docs/get-started/training-on-device.md
Normal file
48
docs/get-started/training-on-device.md
Normal file
|
|
@ -0,0 +1,48 @@
|
|||
---
|
||||
title: On-Device Training
|
||||
parent: Get Started
|
||||
nav_order: 12
|
||||
---
|
||||
|
||||
# On-Device Training with ONNX Runtime
|
||||
{: .no_toc }
|
||||
`On-Device Training` refers to the process of training a model on an edge device, such as mobile phones, embedded devices, gaming consoles, web browsers, etc. This is in contrast to training a model on a server or a cloud. Training on the device can be used for:
|
||||
|
||||
- Personalization tasks, where the model needs to be trained on the user's data.
|
||||
- Federated learning tasks, where the model is locally trained on data that is distributed across multiple devices in an effort to build a more robust aggregated global model.
|
||||
- Improving data privacy and security, especially when working with sensitive data that cannot be shared with a server or a cloud.
|
||||
- Training locally (without impacting application functionality) when network connectivity is unreliable or limited.
|
||||
|
||||
|
||||
`ONNX Runtime Training` offers an easy way to efficiently train and infer ONNX models on edge devices. The training process is divided into two phases:
|
||||
- [the offline phase](#the-offline-phase)
|
||||
- [the training phase](#the-training-phase).
|
||||
|
||||
## The Offline Phase
|
||||
In this phase, training artifacts are prepared on a server, cloud or a desktop that does not have access to user data. These artifacts can be generated by using the `ONNX Runtime Training`'s [artifact generation Python tools](./../api/python/on_device_training/training_artifacts.html) available in the Python package.
|
||||
|
||||
Refer to the [installation instructions](./../install/index.md#offline-phase---prepare-for-training)
|
||||
|
||||
## The Training Phase
|
||||
Once these artifacts are generated, they can be deployed to production scenarios on edge devices. `ONNX Runtime` offers a wide range of packages in multiple language bindings.
|
||||
|
||||
Refer to the [installation instructions](./../install/index.md#training-phase---on-device-training) for a complete list of all language bindings.
|
||||
|
||||
Once training on the edge device is complete, an inference-ready ONNX model can be generated on the edge device itself. This model can then be used with ONNX Runtime for inferencing.
|
||||
|
||||
[comment]: <> (Learn more from the blogs. Links to the blogs go here.)
|
||||
|
||||
## Installation
|
||||
|
||||
Refer to the [installation instructions](./../install/index.md#install-for-on-device-training) for details on how to install for your scenario.
|
||||
|
||||
## Building from Source
|
||||
|
||||
Refer to the [build instructions](./../build/training.md#build-for-on-device-training) for details on how to build for your custom scenario.
|
||||
|
||||
[comment]: <> (Learn more from the tutorials. Links to the demo and website tutorial go here.)
|
||||
[comment]: <> (Also link to the onnxruntime-training-examples repo goes here.)
|
||||
|
||||
## Feature Request, Bug Report or Help Needed
|
||||
|
||||
In case you need help, please open an [issue](https://github.com/microsoft/onnxruntime/issues/new?assignees=&labels=training&projects=&template=06-training.yml&title=%5BTraining%5D+).
|
||||
|
|
@ -1,33 +1,38 @@
|
|||
---
|
||||
title: ORT Training with PyTorch
|
||||
title: Large Model Training
|
||||
parent: Get Started
|
||||
nav_order: 12
|
||||
---
|
||||
|
||||
# Get started with ORT for Training API (PyTorch)
|
||||
# Get started with Large Model Training with ORTModule
|
||||
{: .no_toc }
|
||||
The ORT Training API is a PyTorch frontend that implements the torch.nn.Module interface.
|
||||
|
||||
`ONNX Runtime Training`'s `ORTModule` offers a high performance training engine for models defined using the `PyTorch` frontend. `ORTModule` is designed to accelerate the training of large models without needing to change the model definition and with just a single line of code change (the `ORTModule` wrap) to the entire training script.
|
||||
|
||||
Using the ORTModule class wrapper, ONNX Runtime runs the forward and backward pass of the training script using an optimized automatically-exported ONNX computation graph.
|
||||
|
||||
## ORT Training Example
|
||||
In this example we will go over how to use ORT for Training a model with PyTorch.
|
||||
|
||||
```
|
||||
```sh
|
||||
# Installs the torch_ort and onnxruntime-training Python packages
|
||||
pip install torch-ort
|
||||
# Configures onnxruntime-training to work with user's PyTorch installation
|
||||
python -m torch_ort.configure
|
||||
```
|
||||
|
||||
**Note**: This installs the default version of the `torch-ort` and `onnxruntime-training` packages that are mapped to specific versions of the CUDA libraries. Refer to the install options in [ONNXRUNTIME.ai](https://onnxruntime.ai).
|
||||
**Note**: This installs the default version of the `torch-ort` and `onnxruntime-training` packages that are mapped to specific versions of the CUDA libraries. Refer to the install options in [onnxruntime.ai](https://onnxruntime.ai).
|
||||
|
||||
- Add ORTModule in the `train.py`
|
||||
```python
|
||||
from torch_ort import ORTModule
|
||||
.
|
||||
.
|
||||
.
|
||||
model = ORTModule(model)
|
||||
```
|
||||
|
||||
```diff
|
||||
+ from torch_ort import ORTModule
|
||||
.
|
||||
.
|
||||
.
|
||||
- model = build_model() # Users PyTorch model
|
||||
+ model = ORTModule(build_model())
|
||||
```
|
||||
|
||||
## Samples
|
||||
[ONNX Runtime Training Examples](https://github.com/microsoft/onnxruntime-training-examples)
|
||||
|
|
@ -20,7 +20,8 @@ nav_order: 3
|
|||
| [Microsoft.ML.OnnxRuntime](https://www.nuget.org/packages/Microsoft.ML.OnnxRuntime) | CPU (Release) |Windows, Linux, Mac, X64, X86 (Windows-only), ARM64 (Windows-only)...more details: [compatibility](../reference/compatibility) |
|
||||
| [Microsoft.ML.OnnxRuntime.Gpu](https://www.nuget.org/packages/Microsoft.ML.OnnxRuntime.gpu) | GPU - CUDA (Release) | Windows, Linux, Mac, X64...more details: [compatibility](../reference/compatibility) |
|
||||
| [Microsoft.ML.OnnxRuntime.DirectML](https://www.nuget.org/packages/Microsoft.ML.OnnxRuntime.directml) | GPU - DirectML (Release) | Windows 10 1709+ |
|
||||
| [ort-nightly](https://aiinfra.visualstudio.com/PublicPackages/_packaging?_a=feed&feed=ORT-Nightly) | CPU, GPU (Dev) | Same as Release versions |
|
||||
| [ort-nightly](https://aiinfra.visualstudio.com/PublicPackages/_packaging?_a=feed&feed=ORT-Nightly) | CPU, GPU (Dev), CPU (On-Device Training) | Same as Release versions |
|
||||
| [Microsoft.ML.OnnxRuntime.Training](https://www.nuget.org/packages/Microsoft.ML.OnnxRuntime) | CPU On-Device Training (Release) |Windows, Linux, Mac, X64, X86 (Windows-only), ARM64 (Windows-only)...more details: [compatibility](../reference/compatibility) |
|
||||
|
||||
|
||||
.zip and .tgz files are also included as assets in each [Github release](https://github.com/microsoft/onnxruntime/releases).
|
||||
|
|
|
|||
|
|
@ -20,7 +20,8 @@ nav_order: 2
|
|||
| [Microsoft.ML.OnnxRuntime](https://www.nuget.org/packages/Microsoft.ML.OnnxRuntime) | CPU (Release) |Windows, Linux, Mac, X64, X86 (Windows-only), ARM64 (Windows-only)...more details: [compatibility](../reference/compatibility.md) |
|
||||
| [Microsoft.ML.OnnxRuntime.Gpu](https://www.nuget.org/packages/Microsoft.ML.OnnxRuntime.gpu) | GPU - CUDA (Release) | Windows, Linux, Mac, X64...more details: [compatibility](../reference/compatibility.md) |
|
||||
| [Microsoft.ML.OnnxRuntime.DirectML](https://www.nuget.org/packages/Microsoft.ML.OnnxRuntime.directml) | GPU - DirectML (Release) | Windows 10 1709+ |
|
||||
| [ort-nightly](https://aiinfra.visualstudio.com/PublicPackages/_packaging?_a=feed&feed=ORT-Nightly) | CPU, GPU (Dev) | Same as Release versions |
|
||||
| [ort-nightly](https://aiinfra.visualstudio.com/PublicPackages/_packaging?_a=feed&feed=ORT-Nightly) | CPU, GPU (Dev), CPU (On-Device Training) | Same as Release versions |
|
||||
| [Microsoft.ML.OnnxRuntime.Training](https://www.nuget.org/packages/Microsoft.ML.OnnxRuntime) | CPU On-Device Training (Release) |Windows, Linux, Mac, X64, X86 (Windows-only), ARM64 (Windows-only)...more details: [compatibility](../reference/compatibility.md) |
|
||||
|
||||
.zip and .tgz files are also included as assets in each [Github release](https://github.com/microsoft/onnxruntime/releases).
|
||||
|
||||
|
|
|
|||
|
|
@ -103,8 +103,9 @@ using (var outputs1 = session1.Run(inputs1))
|
|||
|
||||
If the model have fixed sized inputs and outputs of numeric tensors, you can use "FixedBufferOnnxValue" to accelerate the inference speed. By using "FixedBufferOnnxValue", the container objects only need to be allocated/disposed one time during multiple InferenceSession.Run() calls. This avoids some overhead which may be beneficial for smaller models where the time is noticeable in the overall running time.
|
||||
|
||||
An example can be found at `TestReusingFixedBufferOnnxValueNonStringTypeMultiInferences()`:
|
||||
* [Microsoft.ML.OnnxRuntime.Tests/InferenceTest.cs#L1047](https://github.com/microsoft/onnxruntime/tree/main/csharp/test/Microsoft.ML.OnnxRuntime.Tests/InferenceTest.cs#L1047)
|
||||
<!-- FIXME!: This test is no longer in the repo. Needs to be fixed. -->
|
||||
<!-- An example can be found at `TestReusingFixedBufferOnnxValueNonStringTypeMultiInferences()`:
|
||||
* [Microsoft.ML.OnnxRuntime.Tests/InferenceTest.cs#L1047](https://github.com/microsoft/onnxruntime/blob/main/csharp/test/Microsoft.ML.OnnxRuntime.Tests.Common/InferenceTest.cs#L1047) -->
|
||||
|
||||
## Running on GPU (Optional)
|
||||
If using the GPU package, simply use the appropriate SessionOptions when creating an InferenceSession.
|
||||
|
|
@ -128,7 +129,8 @@ The ONNX runtime provides a C# .NET binding for running inference on ONNX models
|
|||
| [Microsoft.ML.OnnxRuntime](https://www.nuget.org/packages/Microsoft.ML.OnnxRuntime) | CPU (Release) |Windows, Linux, Mac, X64, X86 (Windows-only), ARM64 (Windows-only)...more details: [compatibility](../reference/compatibility.md) |
|
||||
| [Microsoft.ML.OnnxRuntime.Gpu](https://www.nuget.org/packages/Microsoft.ML.OnnxRuntime.gpu) | GPU - CUDA (Release) | Windows, Linux, Mac, X64...more details: [compatibility](../reference/compatibility.md) |
|
||||
| [Microsoft.ML.OnnxRuntime.DirectML](https://www.nuget.org/packages/Microsoft.ML.OnnxRuntime.directml) | GPU - DirectML (Release) | Windows 10 1709+ |
|
||||
| [ort-nightly](https://aiinfra.visualstudio.com/PublicPackages/_packaging?_a=feed&feed=ORT-Nightly) | CPU, GPU (Dev) | Same as Release versions |
|
||||
| [ort-nightly](https://aiinfra.visualstudio.com/PublicPackages/_packaging?_a=feed&feed=ORT-Nightly) | CPU, GPU (Dev), CPU (On-Device Training) | Same as Release versions |
|
||||
| [Microsoft.ML.OnnxRuntime.Training](https://www.nuget.org/packages/Microsoft.ML.OnnxRuntime) | CPU On-Device Training (Release) |Windows, Linux, Mac, X64, X86 (Windows-only), ARM64 (Windows-only)...more details: [compatibility](../reference/compatibility.md) |
|
||||
|
||||
|
||||
## API Reference
|
||||
|
|
|
|||
|
|
@ -251,10 +251,6 @@ If using pip, run pip install `--upgrade pip` prior to downloading.
|
|||
|
||||
For Python compiler version notes, see [this page](https://github.com/microsoft/onnxruntime/tree/main/docs/Python_Dev_Notes.md)
|
||||
|
||||
|
||||
## Supported Versions
|
||||
Python 3.6 - 3.9
|
||||
|
||||
## Learn More
|
||||
- [Python Tutorials](../tutorials/api-basics)
|
||||
* [TensorFlow with ONNX Runtime](../tutorials/tf-get-started.md)
|
||||
|
|
|
|||
|
|
@ -66,12 +66,5 @@ ONNX Runtime applies a number of graph optimizations on the model graph then par
|
|||
|
||||
## ONNX Runtime for Training
|
||||
|
||||
Released in April 2021, ONNX Runtime Training provides a one-line addition for existing PyTorch training scripts to accelerate training times. The current support is focused on large transformer models on multi-node NVIDIA GPUs, with more to come.
|
||||
|
||||
### How it works
|
||||
{: .no_toc }
|
||||
|
||||
Using the ORTModule class wrapper, ONNX Runtime for PyTorch runs the forward and backward passes of the training script using an optimized automatically-exported ONNX computation graph. ORT Training uses the same graph optimizations as ORT Inferencing, allowing for model training acceleration.
|
||||
|
||||
The ORTModule is instantiated from [`torch-ort`](https://github.com/pytorch/ort) backend in PyTorch. This new interface enables a seamless integration for ONNX Runtime training in a PyTorch training code with minimal changes to the existing code.
|
||||
|
||||
- [Large Model Training](./get-started/training-pytorch.md)
|
||||
- [On-Device Training](./get-started/training-on-device.md)
|
||||
|
|
|
|||
|
|
@ -176,26 +176,111 @@ Download the [onnxruntime-android](https://mvnrepository.com/artifact/com.micros
|
|||
|
||||
Refer to the instructions for creating a [custom Android package](../build/custom.md#android).
|
||||
|
||||
## ORT Training package
|
||||
## Install for On-Device Training
|
||||
|
||||
Unless stated otherwise, the installation instructions in this section refer to pre-built packages designed to perform on-device training.
|
||||
|
||||
If the pre-built training package supports your model but is too large, you can create a [custom training build](../build/custom.md).
|
||||
|
||||
### Offline Phase - Prepare for Training
|
||||
|
||||
```bash
|
||||
pip install onnxruntime-training
|
||||
```
|
||||
|
||||
### Training Phase - On-Device Training
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>Device</th>
|
||||
<th>Language</th>
|
||||
<th>PackageName</th>
|
||||
<th>Installation Instructions</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Windows</td>
|
||||
<td>C, C++, C#</td>
|
||||
<!-- TODO (baijumeswani) - Update to Training link once published -->
|
||||
<td><a href="https://www.nuget.org/packages/Microsoft.ML.OnnxRuntime">Microsoft.ML.OnnxRuntime.Training</a></td>
|
||||
<td>
|
||||
<pre lang="bash">dotnet add package Microsoft.ML.OnnxRuntime.Training</pre>
|
||||
</td>
|
||||
</tr>
|
||||
<!-- <tr>
|
||||
<td></td>
|
||||
<td>Python</td>
|
||||
<td><a href="https://pypi.org/project/onnxruntime-training/">onnxruntime-training</a></td>
|
||||
<td>
|
||||
<pre lang="bash">pip install onnxruntime-training</pre>
|
||||
</td>
|
||||
</tr> -->
|
||||
<tr>
|
||||
<td>Linux</td>
|
||||
<td>C, C++</td>
|
||||
<td><a href="https://github.com/microsoft/onnxruntime/releases">onnxruntime-training-linux*.tgz</a></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li>Download the <code>*.tgz</code> file from <a href="https://github.com/microsoft/onnxruntime/releases">here</a>.</li>
|
||||
<li>Extract it.</li>
|
||||
<li>Move and include the header files in the <code>include</code> directory.</li>
|
||||
<li>Move the <code>libonnxruntime.so</code> dynamic library to a desired path and include it.</li>
|
||||
</ul>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td>
|
||||
<td>Python</td>
|
||||
<td><a href="https://pypi.org/project/onnxruntime-training/">onnxruntime-training</a></td>
|
||||
<td>
|
||||
<pre lang="bash">pip install onnxruntime-training</pre>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Android</td>
|
||||
<td>C, C++</td>
|
||||
<td><a href="https://mvnrepository.com/artifact/com.microsoft.onnxruntime/onnxruntime-training-android">onnxruntime-training-android</a></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li>Download the <a href="https://mvnrepository.com/artifact/com.microsoft.onnxruntime/onnxruntime-android">onnxruntime-training-android (full package)</a> AAR hosted at Maven Central.</li>
|
||||
<li>Change the file extension from <code>.aar</code> to <code>.zip</code>, and unzip it.</li>
|
||||
<li>Include the header files from the <code>headers</code> folder.</li>
|
||||
<li>Include the relevant <code>libonnxruntime.so</code> dynamic library from the <code>jni</code> folder in your NDK project.</li>
|
||||
</ul>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td>
|
||||
<td>Java/Kotlin</td>
|
||||
<td><a href="https://mvnrepository.com/artifact/com.microsoft.onnxruntime/onnxruntime-android">onnxruntime-training-android</a></td>
|
||||
<td>In your Android Studio Project, make the following changes to:
|
||||
<ol>
|
||||
<li>build.gradle (Project):
|
||||
<pre lang="gradle">
|
||||
repositories {
|
||||
mavenCentral()
|
||||
}
|
||||
</pre>
|
||||
</li>
|
||||
<li>build.gradle (Module):
|
||||
<pre lang="gradle">
|
||||
dependencies {
|
||||
implementation 'com.microsoft.onnxruntime:onnxruntime-training-android:latest.release'
|
||||
}
|
||||
</pre>
|
||||
</li>
|
||||
</ol>
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## Large Model Training
|
||||
|
||||
```bash
|
||||
pip install torch-ort
|
||||
python -m torch_ort.configure
|
||||
```
|
||||
|
||||
**Note**: This installs the default version of the `torch-ort` and `onnxruntime-training` packages that are mapped to specific versions of the CUDA libraries. Refer to the install options in [ONNXRUNTIME.ai](https://onnxruntime.ai).
|
||||
|
||||
### Add ORTModule in the `train.py`
|
||||
|
||||
```python
|
||||
from torch_ort import ORTModule
|
||||
.
|
||||
.
|
||||
.
|
||||
model = ORTModule(model)
|
||||
```
|
||||
|
||||
**Note**: the `model` where ORTModule is wrapped needs to be a derived from the `torch.nn.Module` class.
|
||||
**Note**: This installs the default version of the `torch-ort` and `onnxruntime-training` packages that are mapped to specific versions of the CUDA libraries. Refer to the install options in [onnxruntime.ai](https://onnxruntime.ai).
|
||||
|
||||
## Inference install table for all languages
|
||||
|
||||
|
|
@ -234,20 +319,4 @@ In addition to general [requirements](#requirements), please note additional req
|
|||
|
||||
## Training install table for all languages
|
||||
|
||||
ONNX Runtime Training packages are available for different versions of PyTorch, CUDA and ROCm versions.
|
||||
|
||||
The install command is:
|
||||
```cmd
|
||||
pip3 install torch-ort [-f location]
|
||||
python 3 -m torch_ort.configure
|
||||
```
|
||||
|
||||
The _location_ needs to be specified for any specific version other than the default combination. The location for the different configurations are below:
|
||||
|
||||
||Official build (location)|Nightly build (location)|
|
||||
|---|---|---|
|
||||
|PyTorch 1.11 (ROCm 5.2)||[onnxruntime_nightly_torch1110.rocm52](https://download.onnxruntime.ai/onnxruntime_stable_rocm52.html)|
|
||||
|PyTorch 1.12.1 (ROCm 5.2.3)||[onnxruntime_nightly_torch1121.rocm523](https://download.onnxruntime.ai/onnxruntime_nightly_rocm523.html)|
|
||||
|PyTorch 1.13 (ROCm 5.2.3)||[onnxruntime_nightly_torch1130.rocm523](https://download.onnxruntime.ai/onnxruntime_nightly_rocm523.html)|
|
||||
|PyTorch 1.12.1 (ROCm 5.3.2)||[onnxruntime_nightly_torch1121.rocm532](https://download.onnxruntime.ai/onnxruntime_nightly_rocm532.html)|
|
||||
|PyTorch 1.13.1(ROCm 5.4)||[onnxruntime_nightly_torch1131.rocm54](https://download.onnxruntime.ai/onnxruntime_nightly_rocm54.html)|
|
||||
Refer to the getting started with [Optimized Training](../../index.html#getStartedTable) page for more fine-grained installation instructions.
|
||||
|
|
|
|||
|
|
@ -50,7 +50,7 @@ E.g., [aten_op_tests.py](https://github.com/microsoft/onnxruntime/blob/main/onnx
|
|||
|
||||
Shape inference C++ tests should be added in
|
||||
[onnxruntime/test/contrib_ops](https://github.com/microsoft/onnxruntime/tree/main/onnxruntime/test/contrib_ops).
|
||||
E.g., [trilu_shape_inference_test.cc](https://github.com/microsoft/onnxruntime/tree/main/onnxruntime/test/contrib_ops/trilu_shape_inference_test.cc)
|
||||
E.g., [trilu_shape_inference_test.cc](https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/test/providers/cpu/tensor/trilu_shape_inference_test.cc)
|
||||
|
||||
The operator kernel should be implemented using `Compute` function
|
||||
under contrib namespace in [onnxruntime/contrib_ops/cpu/](https://github.com/microsoft/onnxruntime/tree/main/onnxruntime/contrib_ops/cpu/)
|
||||
|
|
@ -86,7 +86,7 @@ Status Inverse::Compute(OpKernelContext* ctx) const {
|
|||
} // namespace onnxruntime
|
||||
```
|
||||
|
||||
The kernel should be registered in [cpu_contrib_kernels.cc](https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/contrib_ops/cpu/cpu_contrib_kernels.cc) for CPU and [cuda_contrib_kernels.cc](https://github.com/microsoft/onnxruntime/tree/main/onnxruntime/cuda/contrib_ops/cuda_contrib_kernels.cc) for CUDA.
|
||||
The kernel should be registered in [cpu_contrib_kernels.cc](https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/contrib_ops/cpu/cpu_contrib_kernels.cc) for CPU and [cuda_contrib_kernels.cc](https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/contrib_ops/cuda/cuda_contrib_kernels.cc) for CUDA.
|
||||
|
||||
Now you should be able to build and install ONNX Runtime to start using your custom op.
|
||||
|
||||
|
|
|
|||
|
|
@ -982,4 +982,4 @@ Console.WriteLine(outputs[0].AsTensor<float>()[0]);</code>
|
|||
<script src="./js/script.js?v1.6"></script>
|
||||
</body>
|
||||
|
||||
</html>
|
||||
</html>
|
||||
Loading…
Reference in a new issue