25125b5b91
Changes include: - Added a new page for `On-Device Training` overview: [Preview](https://baijumeswani.github.io/onnxruntime/docs/get-started/on-device-training.html) - Added a new section for `On-Device Training` installation: [Preview](https://baijumeswani.github.io/onnxruntime/docs/install/#install-for-on-device-training) - Added a new section for `On-Device Training` build from source: [Preview](https://baijumeswani.github.io/onnxruntime/docs/build/training.html#build-for-on-device-training) - Updated Large Model Training overview, installation, build pages to reflect what is currently accurate. Website preview: https://baijumeswani.github.io/onnxruntime/ Pending website work: - Update links for released packages for training. - Add tutorial for on-device training - Add links to the blog posts that detail on device training.
4 KiB
| title | description | has_children | nav_order | redirect_from |
|---|---|---|---|---|
| ONNX Runtime | ONNX Runtime is a cross-platform machine-learning model accelerator | false | 0 | /how-to |
Welcome to ONNX Runtime
{: .no_toc }
ONNX Runtime is a cross-platform machine-learning model accelerator, with a flexible interface to integrate hardware-specific libraries. ONNX Runtime can be used with models from PyTorch, Tensorflow/Keras, TFLite, scikit-learn, and other frameworks.
How to use ONNX Runtime
|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------| | Get started with ORT{: .btn .mr-4 target="_blank"} | API Docs{: .btn target="_blank"} | | Tutorials{: .btn target="_blank"} | Ecosystem{: .btn target="_blank"} | | ONNX Runtime YouTube{: .btn .mr-4 target="_blank"} |
Contribute and Customize
|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------| | Build ORT Packages{: .btn .mr-4 target="_blank"} | ONNX Runtime GitHub{: .btn target="_blank"} |
QuickStart Template
|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------| | ORT Web JavaScript Site Template{: .btn .mr-4 target="_blank"} | ORT C# Console App Template{: .btn .mr-4 target="_blank"} |
ONNX Runtime for Inferencing
ONNX Runtime Inference powers machine learning models in key Microsoft products and services across Office, Azure, Bing, as well as dozens of community projects.
Examples use cases for ONNX Runtime Inferencing include:
- Improve inference performance for a wide variety of ML models
- Run on different hardware and operating systems
- Train in Python but deploy into a C#/C++/Java app
- Train and perform inference with models created in different frameworks
How it works
{: .no_toc }
The premise is simple.
- Get a model. This can be trained from any framework that supports export/conversion to ONNX format. See the tutorials for some of the popular frameworks/libraries.
- Load and run the model with ONNX Runtime. See the basic tutorials for running models in different languages.
- (Optional) Tune performance using various runtime configurations or hardware accelerators. There are lots of options here - see the Performance section as a starting point.
Even without step 3, ONNX Runtime will often provide performance improvements compared to the original framework.
ONNX Runtime applies a number of graph optimizations on the model graph then partitions it into subgraphs based on available hardware-specific accelerators. Optimized computation kernels in core ONNX Runtime provide performance improvements and assigned subgraphs benefit from further acceleration from each Execution Provider.