[docs] Update mobile performance tuning docs for 1.11 (#11149)

* custom.md - use release tags

* Rename mobile performance tuning doc.

* Move old mobile performance tuning docs.

* remove title

* Adapt original documentation and add more for 1.11.

* Adjust doc ordering.

* Update version text.

* Update nav, fix links.

* rework docs

docs/build/custom.md

@@ -169,16 +169,16 @@ pip install -U .\build\Windows\MinSizeRel\MinSizeRel\dist\onnxruntime-1.7.0-cp37
 
 Unless there is a specific feature you need, do not use the unreleased `master` branch.
 
-Once you have cloned the ONNX Runtime repo, checkout one of the release branches to build from.
+Once you have cloned the ONNX Runtime repo, checkout one of the release tags to build from.
 
 ```bash
 git clone --recursive https://github.com/microsoft/onnxruntime
 cd onnxruntime
-git checkout <release branch>
+git checkout <release tag>
 ```
 
-Release branch names follow the pattern `rel-<version>`. For example, `rel-1.11.0`.
-Find them [here](https://github.com/microsoft/onnxruntime/branches/all?query=rel-).
+Release tag names follow the pattern `v<release version>`. For example, `v1.11.0`.
+Find them [here](https://github.com/microsoft/onnxruntime/tags).
 
 ## Example build commands
 

docs/performance/float16.md

@@ -1,7 +1,7 @@
 ---
 title: Create Float16 and Mixed Precision Models
 parent: How to
-nav_order: 5
+nav_order: 4
 ---
 # Create Float16 and Mixed Precision Models
 {: .no_toc }

docs/performance/graph-optimizations.md

@@ -1,7 +1,7 @@
 ---
 title: Graph optimizations
 parent: Performance
-nav_order: 3
+nav_order: 2
 redirect_from: /docs/how-to/graph-optimizations
 ---
 

docs/performance/mobile-performance-tuning.md

@@ -1,11 +1,12 @@
 ---
 title: Tune Mobile Performance
 parent: Performance
-nav_order: 2
-redirect_from: /docs/how-to/mobile-performance-tuning
+nav_exclude: true
 ---
 {::options toc_levels="2" /}
 
+# **IMPORTANT: THIS INFORMATION ONLY APPLIES TO ONNX RUNTIME VERSION 1.10 AND EARLIER. PLEASE USE A NEWER VERSION.**
+
 # ONNX Runtime Mobile Performance Tuning
 
 Learn how different optimizations affect performance, and get suggestions for performance testing with ORT format models.

docs/performance/quantization.md

@@ -1,7 +1,7 @@
 ---
 title: Quantize ONNX Models
 parent: Performance
-nav_order: 4
+nav_order: 3
 redirect_from: /docs/how-to/quantization
 ---
 # Quantize ONNX Models

docs/reference/mobile/helpers.md

@@ -3,6 +3,7 @@ title: Model Export Helpers
 descriptions: Helpers to assist with export and usage of models with ORT Mobile
 parent: Mobile
 grand_parent: Reference
+nav_order: 1
 ---
 
 # ORT Mobile Model Export Helpers

docs/reference/mobile/ort-format-model-runtime-optimization.md

@@ -0,0 +1,59 @@
+---
+title: ORT Format Model Runtime Optimization
+parent: Mobile
+grand_parent: Reference
+nav_order: 2
+---
+{::options toc_levels="2" /}
+
+# ORT Format Model Runtime Optimization
+
+## Contents
+{: .no_toc }
+
+* TOC placeholder
+{:toc}
+
+## Background
+
+The full ONNX Runtime build supports [graph optimizations](../../performance/graph-optimizations.md) at runtime for ONNX models.
+
+The ORT format model was designed to be used with ONNX Runtime [minimal builds](../../build/custom.md#minimal-build) for environments where smaller binary size is important. To reduce the binary size, some or all of the graph optimizer code is excluded from a minimal build. As such, ONNX models and ORT format models do not share the same graph optimization process.
+
+In ONNX Runtime **1.11 and later**, there is limited support for graph optimizations at runtime for ORT format models. This only applies to extended minimal builds or full builds.
+
+In ONNX Runtime **1.10 and earlier**, there is **no support** for graph optimizations at runtime for ORT format models. Any graph optimizations must be done at model conversion time.
+
+As a rule, [basic graph optimizations](../../performance/graph-optimizations.md#basic-graph-optimizations) are semantics-preserving and result in a valid ONNX graph. The basic optimizations can and generally should be baked in to the converted ORT format model at conversion time - this is the default behavior of the conversion script. In fact, any runtime optimization support for ORT format models will not include basic optimizations at all.
+
+## Types of runtime optimization
+
+These only apply to extended minimal builds or full builds.
+
+### Saved runtime optimizations
+
+Some graph optimizers support additional modes to save and load information about potential graph optimizations to and from the ORT format model. These potential optimizations are known as saved runtime optimizations.
+
+Saved runtime optimizations are only applied at runtime if they are still applicable. For example, a CPU Execution Provider (EP)-specific optimization for some nodes is only applicable if those nodes are assigned to the CPU EP at runtime.
+
+When converting from ONNX to ORT format, the potential optimizations are identified (1) and their effects are saved alongside the graph (without those optimizations applied) in the ORT format model. Later, when loading the ORT format model with saved runtime optimizations, the effects of potential optimizations are applied (2) if the potential optimizations are still applicable.
+
+In an extended minimal build, only enough implementation to support (2) is included, reducing the binary size.
+
+### Graph optimizers
+
+Some graph optimizers are also fully enabled in an extended minimal build and can be directly applied to an ORT format model. One example is the NHWC transformer.
+
+## Choosing whether to use runtime optimizations
+
+The use of runtime optimizations is optional. Generally, the choice is between using an ORT format model with saved runtime optimizations and using a fully optimized ORT format model.
+
+A fully optimized model will have the full set of ONNX Runtime optimizations (at the extended level or higher) available but will be fully optimized for the configuration at model conversion time.
+
+A model with saved runtime optimizations has fewer optimizations available but has more flexibility at runtime. For example, at runtime, a compiling EP like the NNAPI EP can claim the set of nodes it can handle and the remaining nodes can be further optimized with saved runtime optimizations.
+
+You can compare the performance of:
+- A fully optimized model run with only the CPU EP enabled
+- A model with saved runtime optimizations run with additional EPs enabled
+
+The [model usability checker](./model-usability-checker.md) will provide guidance for a particular model.

docs/reference/ort-format-models.md

@@ -22,26 +22,6 @@ The ORT format is the format supported by reduced size ONNX Runtime builds. Redu
 
 Both ORT format models and ONNX models are supported by a full ONNX Runtime build.
 
-### Saved runtime optimizations
-
-**Since ONNX Runtime 1.11**
-
-An ORT format model may contain extra information about potential graph optimizations that may be applied at runtime. These are known as saved runtime optimizations.
-
-Saved runtime optimizations are only applied at runtime if they are still applicable.
-For example, a CPU Execution Provider (EP)-specific optimization for some nodes is only be applicable if those nodes are assigned to the CPU EP at runtime.
-
-In a reduced size ONNX Runtime build, many of the graph optimizers are not supported the way they are in a full build because their complete implementations have significant binary size.
-Consequently, the graph optimizers do not run the same way at runtime for ORT format models as they do for ONNX format models.
-
-To allow for dynamic graph optimization at runtime, some graph optimizers support additional modes to save and load saved runtime optimizations to and from the ORT format model.
-When converting from ONNX to ORT format, the potential optimizations are identified (1) and their effects are saved alongside the graph (without those optimizations applied) in the ORT format model.
-Later, when loading the ORT format model with saved runtime optimizations, the effects of potential optimizations are applied (2) if the potential optimizations are still applicable.
-
-In a reduced size build (an [extended minimal build](../build/custom.md#minimal-build)), only enough implementation to support (2) is included, reducing the graph optimizers' binary size.
-
-Note that saving runtime optimizations is optional. Alternatively, it is possible to save a fully optimized ORT format model, but the graph's optimizations may be hardware-dependent and less flexible at runtime.
-
 ## Convert ONNX models to ORT format
 
 ONNX models are converted to ORT format using the `convert_onnx_models_to_ort` script.
@@ -158,12 +138,7 @@ optional arguments:
 
 **Since ONNX Runtime 1.11**
 
-Style of optimization to perform on the ORT format model. Multiple values may be provided. The conversion will run once for each value. The general guidance is to use models optimized with 'Runtime' style when using NNAPI or CoreML and 'Fixed' style otherwise.
-- 'Fixed': Run optimizations directly before saving the ORT format model. This bakes in any platform-specific optimizations.
-- 'Runtime': Run basic optimizations directly and save certain other optimizations to be applied at runtime if possible. This is useful when using a compiling EP like NNAPI or CoreML that may run an unknown (at model conversion time) number of nodes. The saved optimizations can further optimize nodes not assigned to the compiling EP at runtime.
-  See the documentation on [saved runtime optimizations](#saved-runtime-optimizations).
-
-See the documentation on [performance tuning mobile scenarios](../performance/mobile-performance-tuning.md) for more information.
+Specify whether the converted model will be fully optimized ("Fixed") or have saved runtime optimizations ("Runtime"). Both types of models are produced by default. See [here](./mobile/ort-format-model-runtime-optimization.md) for more information.
 
 This replaces the [optimization level](#optimization-level) option from earlier ONNX Runtime versions.