| cmake | ||
| csharp | ||
| docs | ||
| include/onnxruntime/core | ||
| onnxruntime | ||
| package/rpm | ||
| tools | ||
| .clang-format | ||
| .clang-tidy | ||
| .gitattributes | ||
| .gitignore | ||
| .gitmodules | ||
| build.amd64.1411.bat | ||
| build.bat | ||
| BUILD.md | ||
| build.sh | ||
| CODEOWNERS | ||
| CONTRIBUTING.md | ||
| LICENSE | ||
| README.md | ||
| rename_manylinux.sh | ||
| requirements-dev.txt | ||
| requirements.txt | ||
| setup.py | ||
| ThirdPartyNotices.txt | ||
| VERSION_NUMBER | ||
ONNX Runtime
Introduction
ONNX Runtime is an open-source scoring engine for Open Neural Network Exchange (ONNX) models.
ONNX is an open format for machine learning (ML) models that is supported by various ML and DNN frameworks and tools. This format makes it easier to interoperate between frameworks and to maximize the reach of your hardware optimization investments. Learn more about ONNX on https://onnx.ai or view the Github Repo.
Why use ONNX Runtime
Run any ONNX model
ONNX Runtime provides comprehensive support of the ONNX spec and can be used to run all models based on ONNX v1.2.1 and higher. See ONNX version release details here.
In order to support popular and leading AI models, the runtime stays up-to-date with evolving ONNX operators and functionalities.
Cross Platform
ONNX Runtime offers:
- APIs for Python, C#, and C (experimental)
- Available for Linux, Windows, and Mac
See API documentation and package installation instructions below.
High Performance
You can use ONNX Runtime with both CPU and GPU hardware. You can also plug in additional execution providers to ONNX Runtime. With many graph optimizations and various accelerators, ONNX Runtime can often provide lower latency and higher efficiency compared to other runtimes. This provides smoother end-to-end customer experiences and lower costs from improved machine utilization.
Currently ONNX Runtime supports CUDA and MKL-DNN (with option to build with MKL) for computation acceleration, with more coming soon. To add an execution provider, please refer to this page.
Getting Started
If you need a model:
- Check out the ONNX Model Zoo for ready-to-use pre-trained models.
- To get an ONNX model by exporting from various frameworks, see ONNX Tutorials.
If you already have an ONNX model, just install the runtime for your machine to try it out. One easy way to deploy the model on the cloud is by using Azure Machine Learning. See detailed instructions here.
Installation
APIs and Official Builds
| API Documentation | CPU package | GPU package |
|---|---|---|
| Python | Windows Linux Mac |
Windows Linux |
| C# | Windows Linux - Coming Soon Mac - Coming Soon |
Coming Soon |
| C (experimental) | Coming Soon | Coming Soon |
ONNX Runtime also provides a non ABI C++ API
Build Details
For details on the build configurations and information on how to create a build, see Build ONNX Runtime.
Versioning
See more details on API and ABI Versioning and ONNX Compatibility in Versioning.
Design and Key Features
For an overview of the high level architecture and key decisions in the technical design of ONNX Runtime, see Engineering Design.
ONNX Runtime is built with an extensible design that makes it versatile to support a wide array of models with high performance.
- Add a custom operator/kernel
- Add an execution provider
- Add a new graph transform
- Add a new rewrite rule
Contribute
We welcome your contributions! Please see the contribution guidelines.
Feedback
For any feedback or to report a bug, please file a GitHub Issue.
Code of Conduct
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.