mirror of
https://github.com/saymrwulf/onnxruntime.git
synced 2026-07-17 18:40:28 +00:00
[Website] Add ORT at MS page and doc fixes (#12518)
* add ort at microsoft page * remove openmp references * update EP table * update ort-at-microsoft page * fix link * Change ORT to ONNX Runtime Co-authored-by: Nat Kershaw (MSFT) <nakersha@microsoft.com> Co-authored-by: Nat Kershaw (MSFT) <nakersha@microsoft.com>
This commit is contained in:
Faith Xu
GitHub
parent
80ccd0036c
commit
552b70f82a
14 changed files with 146 additions and 36 deletions
11
about.html
11
about.html
|
|
@ -71,10 +71,17 @@
|
|||
<p class="mb-2">
|
||||
ONNX Runtime is an open source project that is designed to accelerate machine learning across a wide range of frameworks, operating systems, and hardware platforms. It enables acceleration of machine learning inferencing across all of your deployment targets using a single set of <abbr title="Application Program Interface">API</abbr>. ONNX Runtime automatically parses through your model to identify optimization opportunities and provides access to the best hardware acceleration available.
|
||||
</p>
|
||||
<p class="mb-3">
|
||||
<p class="mb-2">
|
||||
ONNX Runtime also offers training acceleration, which incorporates innovations from Microsoft Research and is proven across production workloads like Office 365, Bing and Visual Studio.
|
||||
</p>
|
||||
<a href="https://github.com/microsoft/onnxruntime" target="_blank" class="link ft-20"><span class="link-content">Join us on Github</span><span class="link-arrow fa fa-angle-right"></span></a>
|
||||
<br/> <br/>
|
||||
<p class="mb-2">
|
||||
At Microsoft, ONNX Runtime is used as the primary Machine Learning inferencing solution for products groups. ONNX Runtime serves over 1 trillion daily inferences across over 150 production models covering all task domains.
|
||||
</p>
|
||||
|
||||
<a href="ort-at-microsoft.html" target="_blank" class="link ft-20"><span class="link-content">Learn more about ONNX Runtime at Microsoft</span><span class="link-arrow fa fa-angle-right"></span></a>
|
||||
|
||||
</div>
|
||||
<div class="col-12 col-md-3 d-none d-md-block">
|
||||
<img src="./images/ONNX-Icon.png" alt="ONNX Runtime Logo Icon" class="img-fluid">
|
||||
|
|
@ -128,7 +135,7 @@
|
|||
<div class="col-12">
|
||||
<h2 class="mt-3 mt-md-0">Design principles</h2>
|
||||
<p class="mb-md-0 pr-0 pr-md-5">
|
||||
ONNX Runtime abstracts custom accelerators and runtimes to maximize their benefits across an ONNX model. To do this, ONNX Runtime partitions the ONNX model graph into subgraphs that align with available custom accelerators and runtimes. When operators are not supported by custom accelerators or runtimes, ONNX Runtime provides a default runtime that is used as the fallback execution — ensuring that any model will run.
|
||||
ONNX Runtime abstracts custom accelerators and runtimes to maximize their benefits across an ONNX model. To do this, ONNX Runtime partitions the ONNX model graph into subgraphs that align with available custom accelerators and runtimes. When operators are not supported by custom accelerators or runtimes, ONNX Runtime provides a default runtime that is used as the fallback execution — ensuring that any model will run. <a href="docs/reference/high-level-design.html" target="_blank" class="link">Learn more</a>.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
|
|
|||
|
|
@ -207,10 +207,18 @@ a.link:active .link-arrow{
|
|||
height: 55px;
|
||||
}
|
||||
.customer-logo img{
|
||||
max-width: 100%;
|
||||
padding-right:20px;
|
||||
padding-left:20px;
|
||||
}
|
||||
max-width: 100%;
|
||||
padding-right:10px;
|
||||
padding-left:10px;
|
||||
padding-top:20px;
|
||||
padding-bottom:20px;
|
||||
}
|
||||
|
||||
.ms-logo img{
|
||||
max-width: 100%;
|
||||
padding-right:20px;
|
||||
padding-left:20px;
|
||||
}
|
||||
|
||||
|
||||
div.customer-logo:hover{
|
||||
|
|
|
|||
15
docs/build/inferencing.md
vendored
15
docs/build/inferencing.md
vendored
|
|
@ -161,21 +161,6 @@ Nuget packages are created under <native_build_dir>\nuget-artifacts
|
|||
|
||||
[Reduced Operator Kernel](./custom.md#reduce-operator-kernels) builds allow you to customize the kernels in the build to provide smaller binary sizes.
|
||||
|
||||
### OpenMP (Deprecated)
|
||||
|
||||
#### Build Instructions
|
||||
|
||||
##### Windows
|
||||
|
||||
```powershell
|
||||
.\build.bat --use_openmp
|
||||
```
|
||||
|
||||
##### Linux/macOS
|
||||
|
||||
```bash
|
||||
./build.sh --use_openmp
|
||||
```
|
||||
|
||||
### DebugNodeInputsOutputs
|
||||
OnnxRuntime supports build options for enabling debugging of intermediate tensor shapes and data.
|
||||
|
|
|
|||
|
|
@ -28,12 +28,13 @@ ONNX Runtime supports many different execution providers today. Some of the EPs
|
|||
|
||||
|CPU|GPU|IoT/Edge/Mobile|Other|
|
||||
---|---|---|---
|
||||
|Default CPU|[NVIDIA CUDA](../execution-providers/CUDA-ExecutionProvider.md)|[Intel OpenVINO](../execution-providers/OpenVINO-ExecutionProvider.md)|
|
||||
|[Intel DNNL](../execution-providers/oneDNN-ExecutionProvider.md)|[NVIDIA TensorRT](../execution-providers/TensorRT-ExecutionProvider.md)|[ARM Compute Library](../execution-providers/ACL-ExecutionProvider.md) (*preview*)|[Rockchip NPU](../execution-providers/RKNPU-ExecutionProvider.md) (*preview*)|
|
||||
|[TVM](../execution-providers/TVM-ExecutionProvider.md) (*preview*)|[DirectML](../execution-providers/DirectML-ExecutionProvider.md)|[Android Neural Networks API](../execution-providers/NNAPI-ExecutionProvider.md)|[Xilinx Vitis-AI](../execution-providers/Vitis-AI-ExecutionProvider.md) (*preview*)|
|
||||
|Default CPU|[NVIDIA CUDA](../execution-providers/CUDA-ExecutionProvider.md)|[Intel OpenVINO](../execution-providers/OpenVINO-ExecutionProvider.md)|[Rockchip NPU](../execution-providers/RKNPU-ExecutionProvider.md) (*preview*)|
|
||||
|[Intel DNNL](../execution-providers/oneDNN-ExecutionProvider.md)|[NVIDIA TensorRT](../execution-providers/TensorRT-ExecutionProvider.md)|[ARM Compute Library](../execution-providers/ACL-ExecutionProvider.md) (*preview*)|[Xilinx Vitis-AI](../execution-providers/Vitis-AI-ExecutionProvider.md) (*preview*)|
|
||||
|[TVM](../execution-providers/TVM-ExecutionProvider.md) (*preview*)|[DirectML](../execution-providers/DirectML-ExecutionProvider.md)|[Android Neural Networks API](../execution-providers/NNAPI-ExecutionProvider.md)||
|
||||
|[Intel OpenVINO](../execution-providers/OpenVINO-ExecutionProvider.md)|[AMD MIGraphX](../execution-providers/MIGraphX-ExecutionProvider.md) (*preview*)|[ARM-NN](../execution-providers/ArmNN-ExecutionProvider.md) (*preview*)|
|
||||
||[TVM](../execution-providers/TVM-ExecutionProvider.md) (*preview*)|[CoreML](../execution-providers/CoreML-ExecutionProvider.md) (*preview*)|
|
||||
||[Intel OpenVINO](../execution-providers/OpenVINO-ExecutionProvider.md)|[TVM](../execution-providers/TVM-ExecutionProvider.md) (*preview*)|
|
||||
|||[Qualcomm SNPE](../execution-providers/SNPE-ExecutionProvider.md)
|
||||
|
||||
### Add an Execution Provider
|
||||
|
||||
|
|
|
|||
|
|
@ -166,12 +166,9 @@ Currently, there are no special provisions to employ mimalloc on Linux. It is re
|
|||
|
||||
### Thread management
|
||||
|
||||
* If ORT is built with OpenMP, use the OpenMP environment variable to control the number of intra op num threads.
|
||||
* If ORT is not built with OpenMP, use the appropriate ORT API to control intra op num threads.
|
||||
* Inter op num threads (used only when parallel execution is enabled) is not affected by OpenMP settings and should
|
||||
always be set using the ORT APIs.
|
||||
* Use the appropriate ORT API to set intra and inter op num threads. Inter op num threads is only used when parallel execution is enabled.
|
||||
|
||||
### Custom threading callbacks
|
||||
#### Custom threading callbacks
|
||||
|
||||
Occasionally, customers might prefer to use their own fine-tuned threads for multithreading,
|
||||
hence ORT offers thread creation and joining callbacks in the [C++ API](https://github.com/microsoft/onnxruntime/blob/master/include/onnxruntime/core/session/onnxruntime_cxx_api.h):
|
||||
|
|
@ -514,13 +511,8 @@ Will be supported in future releases
|
|||
|
||||
## Troubleshooting performance issues
|
||||
|
||||
The answers below are troubleshooting suggestions based on common previous user-filed issues and questions. This list is by no means exhaustive and there is a lot of case-by-case fluctuation depending on the model and specific usage scenario. Please use this information to guide your troubleshooting, search through previously filed issues for related topics, and/or file a new issue if your problem is still not resolved.
|
||||
Here is a list of things to check when assessing performance issues:
|
||||
|
||||
### Performance Troubleshooting Checklist
|
||||
|
||||
Here is a list of things to check when assessing performance issues.
|
||||
|
||||
* Are you using OpenMP? OpenMP will parallelize some of the code for potential performance improvements. This is not recommended for running on single threads.
|
||||
* Have you enabled all [graph optimizations](graph-optimizations.md)? The official published packages do enable all by default but when building from source, check that these are enabled in your build.
|
||||
* Have you searched through prior-filed [GitHub issues](https://github.com/microsoft/onnxruntime/issues) to see if your problem has been discussed previously? Please do this before filing new issues.
|
||||
* If using CUDA or TensorRT, do you have the right versions of the dependent libraries installed?
|
||||
|
|
@ -543,7 +535,7 @@ NCHW and NHWC are two different memory layout for 4-D tensors.
|
|||
|
||||
Most TensorFlow operations used by a CNN support both NHWC and NCHW data format. The TensorFlow team suggests that on GPUs NCHW is faster but on CPUs NHWC is sometimes faster in TensorFlow. However, ONNX only supports NCHW. As a result, if the original model is in NHWC format, extra transposes may be added when the model is converted. The [tensorflow-onnx](https://github.com/onnx/tensorflow-onnx) converter does remove many of these transposes, but if this doesn't help sufficiently, consider retraining the model in the NCHW format.
|
||||
|
||||
### Mitigate high latency variance
|
||||
### I am seeing high latency variance
|
||||
|
||||
On some platforms, onnxruntime may exhibit high latency variance during inferencing. This is caused by the constant cost model that onnxruntime uses to parallelize tasks in the thread pool.
|
||||
For each task, the constant cost model will calculate a granularity for parallelization among threads, which stays constant to the end of the task execution. This approach can bring imbalanced load sometimes, causing high latency variance.
|
||||
|
|
|
|||
BIN
images/logos/microsoft/azure.png
Normal file
BIN
images/logos/microsoft/azure.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 16 KiB |
BIN
images/logos/microsoft/bing.png
Normal file
BIN
images/logos/microsoft/bing.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 15 KiB |
BIN
images/logos/microsoft/dynamics.png
Normal file
BIN
images/logos/microsoft/dynamics.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 27 KiB |
BIN
images/logos/microsoft/edge.png
Normal file
BIN
images/logos/microsoft/edge.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 16 KiB |
BIN
images/logos/microsoft/office.png
Normal file
BIN
images/logos/microsoft/office.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 17 KiB |
BIN
images/logos/microsoft/powerplatform.png
Normal file
BIN
images/logos/microsoft/powerplatform.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 20 KiB |
BIN
images/logos/microsoft/visualstudio.png
Normal file
BIN
images/logos/microsoft/visualstudio.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 25 KiB |
BIN
images/logos/microsoft/windows.png
Normal file
BIN
images/logos/microsoft/windows.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 11 KiB |
117
ort-at-microsoft.html
Normal file
117
ort-at-microsoft.html
Normal file
|
|
@ -0,0 +1,117 @@
|
|||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
|
||||
<head>
|
||||
<script async src="https://www.googletagmanager.com/gtag/js?id=UA-156955408-1"></script>
|
||||
<script>
|
||||
window.dataLayer = window.dataLayer || [];
|
||||
function gtag(){dataLayer.push(arguments);}
|
||||
gtag('js', new Date());
|
||||
gtag('config', 'UA-156955408-1');
|
||||
</script>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
|
||||
<title>ONNX Runtime | ORT at Microsoft</title>
|
||||
<link rel="icon" href="./images/ONNXRuntime-Favicon.png" type="image/gif" sizes="16x16">
|
||||
<link rel="stylesheet" href="css/fonts.css">
|
||||
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.3.1/css/bootstrap.min.css">
|
||||
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css">
|
||||
<link rel="stylesheet" href="css/custom.css">
|
||||
<link rel="stylesheet" href="css/responsive.css">
|
||||
</head>
|
||||
|
||||
<body>
|
||||
<a class="skip-main" href="#skipMain">Skip to main content</a>
|
||||
<div class="main-wrapper">
|
||||
<div class="top-banner-bg">
|
||||
<header class="fixed-top header-content">
|
||||
<nav class="navbar navbar-expand-md navbar-custom" aria-label="Main menu">
|
||||
<a id="ONNXLogo" class="navbar-brand" href="./index.html">
|
||||
<img src="images/svg/ONNX-Runtime-logo.svg" class="d-inline-block align-top onnx-logo" alt="ONNX Runtime Home" />
|
||||
</a>
|
||||
<button class="navbar-toggler p-0" type="button" data-toggle="collapse" data-target="#navbarNav" aria-controls="navbarNav" aria-expanded="false" aria-label="Toggle navigation">
|
||||
<span class="navbar-toggler-icon"></span>
|
||||
</button>
|
||||
<div class="collapse navbar-collapse border-md-top mt-md-0 mt-2" id="navbarNav">
|
||||
<div class="mr-auto"></div>
|
||||
<div class="my-md-2 mb-0 mt-2 my-lg-0 pl-3 pl-md-0">
|
||||
<ul class="navbar-nav navbar-nav mr-auto text-uppercase" id="navigation">
|
||||
<li class="nav-item">
|
||||
<a class="nav-link pr-3" href="./index.html#getStartedTable">Get Started</a>
|
||||
</li>
|
||||
<li class="nav-item">
|
||||
<a class="nav-link pr-3" href="./docs">Docs</a>
|
||||
</li>
|
||||
<li class="nav-item">
|
||||
<a class="nav-link pr-3" target="_blank" href="./news.html">News</a>
|
||||
</li>
|
||||
<li class="nav-item">
|
||||
<a class="nav-link pr-3" href="./community.html">Community</a>
|
||||
</li>
|
||||
<li class="nav-item">
|
||||
<a class="nav-link pr-3" href="./about.html">About</a>
|
||||
</li>
|
||||
<li class="nav-item">
|
||||
<a class="nav-link" target="_blank" href="http://github.com/microsoft/onnxruntime">GitHub</a>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
</nav>
|
||||
</header>
|
||||
|
||||
<div id="skipMain" role="main">
|
||||
<div class="outer-container mx-auto py-5">
|
||||
|
||||
<section class="blue-title-columns py-md-5 pb-4 pt-4 mt-5">
|
||||
<h1 class="mb-3 blue-text">ONNX Runtime usage at Microsoft</h1>
|
||||
ONNX Runtime powers Machine Learning inferencing for most of Microsoft's products and services, providing high performance and deployment versatility to support a large range of device types across cloud, mobile, and edge.
|
||||
<br/><br/>
|
||||
|
||||
<div class="row equalHeight-1 blue-title-columns">
|
||||
<div class="ms-logo col-4 col-md-4 text-center">
|
||||
<img src="./images/logos/microsoft/azure.png" alt="Azure logo">
|
||||
</div>
|
||||
<div class="ms-logo col-4 col-md-4 text-center">
|
||||
<img src="./images/logos/microsoft/office.png" alt="Office logo">
|
||||
</div>
|
||||
<div class="ms-logo col-4 col-md-4 text-center">
|
||||
<img src="./images/logos/microsoft/bing.png" alt="Bing logo">
|
||||
</div>
|
||||
<div class="ms-logo col-4 col-md-4 text-center">
|
||||
<img src="./images/logos/microsoft/dynamics.png" alt="Dynamics 365 logo">
|
||||
</div>
|
||||
<div class="ms-logo col-4 col-md-4 text-center">
|
||||
<img src="./images/logos/microsoft/windows.png" alt="Windows logo">
|
||||
</div>
|
||||
<div class="ms-logo col-4 col-md-4 text-center">
|
||||
<img src="./images/logos/microsoft/powerplatform.png" alt="Power Platform logo">
|
||||
</div>
|
||||
<div class="ms-logo col-4 col-md-4 text-center">
|
||||
<img src="./images/logos/microsoft/edge.png" alt="Edge browser logo">
|
||||
</div>
|
||||
<div class="ms-logo col-4 col-md-4 text-center">
|
||||
<img src="./images/logos/microsoft/visualstudio.png" alt="Visual Studio logo">
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<!-- Partial footer.html Start-->
|
||||
<div w3-include-html="footer.html"></div>
|
||||
<!-- Partial footer.html End-->
|
||||
|
||||
<a id="back-to-top" href="JavaScript:void(0);" class="btn btn-lg back-to-top" role="button" aria-label="Back to top"><span class="fa fa-angle-up"></span></a>
|
||||
|
||||
<script src="https://www.w3schools.com/lib/w3.js"></script>
|
||||
<script>w3.includeHTML();</script>
|
||||
<script src="https://code.jquery.com/jquery-3.4.1.min.js"></script>
|
||||
<script src="https://maxcdn.bootstrapcdn.com/bootstrap/4.3.1/js/bootstrap.min.js"></script>
|
||||
<script src="./js/custom.js"></script>
|
||||
|
||||
</body>
|
||||
|
||||
</html>
|
||||
Loading…
Reference in a new issue