mirror of
https://github.com/saymrwulf/onnxruntime.git
synced 2026-07-15 18:23:41 +00:00
25125b5b91
Changes include: - Added a new page for `On-Device Training` overview: [Preview](https://baijumeswani.github.io/onnxruntime/docs/get-started/on-device-training.html) - Added a new section for `On-Device Training` installation: [Preview](https://baijumeswani.github.io/onnxruntime/docs/install/#install-for-on-device-training) - Added a new section for `On-Device Training` build from source: [Preview](https://baijumeswani.github.io/onnxruntime/docs/build/training.html#build-for-on-device-training) - Updated Large Model Training overview, installation, build pages to reflect what is currently accurate. Website preview: https://baijumeswani.github.io/onnxruntime/ Pending website work: - Update links for released packages for training. - Add tutorial for on-device training - Add links to the blog posts that detail on device training.
70 lines
4 KiB
Markdown
70 lines
4 KiB
Markdown
---
|
|
title: ONNX Runtime
|
|
description: ONNX Runtime is a cross-platform machine-learning model accelerator
|
|
has_children: false
|
|
nav_order: 0
|
|
redirect_from: /how-to
|
|
---
|
|
|
|
# Welcome to ONNX Runtime
|
|
{: .no_toc }
|
|
|
|
ONNX Runtime is a cross-platform machine-learning model accelerator, with a flexible interface to integrate hardware-specific libraries. ONNX Runtime can be used with models from PyTorch, Tensorflow/Keras, TFLite, scikit-learn, and other frameworks.
|
|
|
|
<iframe height="315" class="table-wrapper py px" src="https://www.youtube.com/embed/waIeC3OIn70" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
|
|
|
|
## How to use ONNX Runtime
|
|
|
|
|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
|
|
| <span class="fs-5"> [Get started with ORT](./get-started){: .btn .mr-4 target="_blank"} </span> | <span class="fs-5"> [API Docs](./api){: .btn target="_blank"} </span> |
|
|
| <span class="fs-5"> [Tutorials](./tutorials){: .btn target="_blank"} </span> | <span class="fs-5"> [Ecosystem](./ecosystem){: .btn target="_blank"} </span> |
|
|
| <span class="fs-5">[ONNX Runtime YouTube](https://www.youtube.com/channel/UC_SJk17KdRvDulXz-nc1uFg/featured){: .btn .mr-4 target="_blank"} </span> |
|
|
|
|
## Contribute and Customize
|
|
|
|
|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
|
|
| <span class="fs-5"> [Build ORT Packages](./build){: .btn .mr-4 target="_blank"} </span>| <span class="fs-5">[ONNX Runtime GitHub](https://github.com/microsoft/onnxruntime){: .btn target="_blank"} </span> |
|
|
|
|
|
|
---
|
|
|
|
## QuickStart Template
|
|
|
|
|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
|
|
| <span class="fs-5"> [ORT Web JavaScript Site Template](https://github.com/microsoft/onnxruntime-nextjs-template){: .btn .mr-4 target="_blank"} </span> | <span class="fs-5"> [ORT C# Console App Template](https://github.com/microsoft/onnxruntime-csharp-cv-template){: .btn .mr-4 target="_blank"} </span> |
|
|
|
|
|
|
---
|
|
|
|
|
|
## ONNX Runtime for Inferencing
|
|
|
|
ONNX Runtime Inference powers machine learning models in key Microsoft products and services across Office, Azure, Bing, as well as dozens of community projects.
|
|
|
|
Examples use cases for ONNX Runtime Inferencing include:
|
|
|
|
* Improve inference performance for a wide variety of ML models
|
|
* Run on different hardware and operating systems
|
|
* Train in Python but deploy into a C#/C++/Java app
|
|
* Train and perform inference with models created in different frameworks
|
|
|
|
### How it works
|
|
{: .no_toc }
|
|
|
|
The premise is simple.
|
|
1. **Get a model.** This can be trained from any framework that supports export/conversion to ONNX format. See the [tutorials](./tutorials) for some of the popular frameworks/libraries.
|
|
2. **Load and run the model with ONNX Runtime.** See the [basic tutorials](./tutorials/api-basics) for running models in different languages.
|
|
3. ***(Optional)* Tune performance using various runtime configurations or hardware accelerators.** There are lots of options here - see the [Performance section](./performance) as a starting point.
|
|
|
|
Even without step 3, ONNX Runtime will often provide performance improvements compared to the original framework.
|
|
|
|
ONNX Runtime applies a number of graph optimizations on the model graph then partitions it into subgraphs based on available hardware-specific accelerators. Optimized computation kernels in core ONNX Runtime provide performance improvements and assigned subgraphs benefit from further acceleration from each [Execution Provider](./execution-providers).
|
|
|
|
|
|
|
|
---
|
|
|
|
## ONNX Runtime for Training
|
|
|
|
- [Large Model Training](./get-started/training-pytorch.md)
|
|
- [On-Device Training](./get-started/training-on-device.md)
|