a4a459477a
* added packaging pipeline * Update win-ci-pipeline.yml for Azure Pipelines * Update win-ci-pipeline.yml for Azure Pipelines * Update win-ci-pipeline.yml for Azure Pipelines * Update win-ci-pipeline.yml for Azure Pipelines * Update win-ci-pipeline.yml for Azure Pipelines * Update win-ci-pipeline.yml for Azure Pipelines * Update win-ci-pipeline.yml for Azure Pipelines * Update win-ci-pipeline.yml for Azure Pipelines * put the c-api header file at root instead of under core/session * Update win-ci-pipeline.yml for Azure Pipelines * Update win-ci-pipeline.yml for Azure Pipelines * Update win-ci-pipeline.yml for Azure Pipelines * parameterize the windows build script * Update win-package-pipeline.yml for Azure Pipelines * fixed indenting * fixed indenting * fix parameter reference syntax * try using arch = amd64 for the vcvarsall * remove duplicate tasks * use vcvarsall * some more refactor * fix typo * fix typo * factored out the packaging step into a template * add x86 build to package pipeline * use amd64 for vcvars arg * added gpu pipeline. added msbuild platform param * fix the msbuild platform * use amd64 host for x86 build * use buildarch=x86 for vcvarsall * remove vcvars from setup steps * add some logging for PNG lib, and disable fns_candy demo for win32 * set allocator alignment to 32 bit for win32 compiler * disable parallel execution test for x86 * use 64 bit toolchain for x86 build * add missing -T flag for toolset * fix string delimietr in workingdirectory name for package build test step * fix gpu pipeline * make io_types test conditional * use cuda 10 instead of cuda 9.1, similar to the ci build * try some workaround on the io test * undo inadvertent local change in build.py, also reenable the io test * make all test run single threaded * blacklist few failing tests for x86 * added some log in build.py * edit build.py to disable parallel test * add the failed tests into the blacklist for win32 * add tf_pasnet_large to blacklist * change control flow for build.py onnx tests * add README, license and TPN to the package * updated build.py test sequence for parallel executor * updated onnx test flow as per review comment * add type checking log in the compare_mlvalue * fix type cast * blacklist some failed test as of now * one more blacklisted test |
||
|---|---|---|
| .github/ISSUE_TEMPLATE | ||
| cmake | ||
| csharp | ||
| dockerfiles | ||
| docs | ||
| include/onnxruntime/core | ||
| onnxruntime | ||
| package/rpm | ||
| tools | ||
| .clang-format | ||
| .clang-tidy | ||
| .gitattributes | ||
| .gitignore | ||
| .gitmodules | ||
| build.amd64.1411.bat | ||
| build.bat | ||
| BUILD.md | ||
| build.sh | ||
| cgmanifest.json | ||
| CODEOWNERS | ||
| CONTRIBUTING.md | ||
| LICENSE | ||
| README.md | ||
| rename_manylinux.sh | ||
| requirements-dev.txt | ||
| requirements-doc.txt | ||
| requirements.txt | ||
| setup.py | ||
| ThirdPartyNotices.txt | ||
| VERSION_NUMBER | ||
ONNX Runtime is an open-source scoring engine for Open Neural Network Exchange (ONNX) models.
ONNX is an open format for machine learning (ML) models that is supported by various ML and DNN frameworks and tools. This format makes it easier to interoperate between frameworks and to maximize the reach of your hardware optimization investments. Learn more about ONNX on https://onnx.ai or view the Github Repo.
Why use ONNX Runtime
ONNX Runtime is an open architecture that is continually evolving to adapt to and address the newest developments and challenges in AI and Deep Learning. We will keep ONNX Runtime up to date with the ONNX standard, supporting all ONNX releases with future compatibliity while maintaining backwards compatibility with prior releases.
ONNX Runtime continuously strives to provide top performance for a broad and growing number of usage scenarios in Machine Learning. Our investments focus on these 3 core areas:
- Run any ONNX model
- High performance
- Cross platform
Run any ONNX model
Alignment with ONNX Releases
ONNX Runtime provides comprehensive support of the ONNX spec and can be used to run all models based on ONNX v1.2.1 and higher. See ONNX version release details here.
As of January 2019, ONNX Runtime supports ONNX 1.3. We will soon add support for the recently released ONNX 1.4.
Traditional ML support
ONNX Runtime fully supports the ONNX-ML profile of the ONNX spec for traditional ML scenarios.
High Performance
You can use ONNX Runtime with both CPU and GPU hardware. You can also plug in additional execution providers to ONNX Runtime. With many graph optimizations and various accelerators, ONNX Runtime can often provide lower latency and higher efficiency compared to other runtimes. This provides smoother end-to-end customer experiences and lower costs from improved machine utilization.
Currently ONNX Runtime supports CUDA and MKL-DNN (with option to build with MKL) for computation acceleration. To add an execution provider, please refer to this page.
We are continuously working to integrate new execution providers to provide improvements in latency and efficiency. We have ongoing collaborations to integrate the following with ONNX Runtime: * Intel MKL-DNN and nGraph * NVIDIA TensorRT
Cross Platform
ONNX Runtime offers:
- APIs for Python, C#, and C
- Available for Linux, Windows, and Mac
See API documentation and package installation instructions below.
Looking ahead: To broaden the reach of the runtime, we will continue investments to make ONNX Runtime available and compatible with more platforms. These include but are not limited to:
- C# for Mac
- ARM
Getting Started
If you need a model:
- Check out the ONNX Model Zoo for ready-to-use pre-trained models.
- To get an ONNX model by exporting from various frameworks, see ONNX Tutorials.
If you already have an ONNX model, just install the runtime for your machine to try it out. One easy way to deploy the model on the cloud is by using Azure Machine Learning. See detailed instructions here.
Installation
APIs and Official Builds
| API Documentation | CPU package | GPU package |
|---|---|---|
| Python | Windows/Linux/Mac | Windows/Linux (Compatible with CUDA9.1 and cuDNN 7.3) |
| C# | Windows/Linux Mac - Coming Soon |
Windows/Linux Mac - Coming Soon |
| C | Windows/Linux Mac - Coming Soon |
Windows/Linux Mac - Coming Soon |
| C++ | TBD | TBD |
System Requirements
- The OnnxRuntime binaries in CPU packages use OpenMP and depends on the library being available at runtime in the system. For Windows, OpenMP support comes as part of VC runtime. For Linux, the system must have the libgomp.so.1 installed.
- The GPU builds require the CUDA9.1 and cuDNN 7.3 runtime libraries being installed in the system.
Build Details
For details on the build configurations and information on how to create a build, see Build ONNX Runtime.
Versioning
See more details on API and ABI Versioning and ONNX Compatibility in Versioning.
Design and Key Features
For an overview of the high level architecture and key decisions in the technical design of ONNX Runtime, see Engineering Design.
ONNX Runtime is built with an extensible design that makes it versatile to support a wide array of models with high performance.
- Add a custom operator/kernel
- Add an execution provider
- Add a new graph transform
- Add a new rewrite rule
Contribute
We welcome your contributions! Please see the contribution guidelines.
Feedback
For any feedback or to report a bug, please file a GitHub Issue.
Code of Conduct
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.