saymrwulf/onnxruntime
1
0
Fork 0
mirror of https://github.com/saymrwulf/onnxruntime.git synced 2026-07-17 18:40:28 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Fix merge conflicts (#9104)

Browse source
This commit is contained in:
Chi Lo 2021-09-17 14:01:15 -07:00 committed by GitHub
parent b65ccc57a7
commit 603ea0e2cf
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
8 changed files with 36 additions and 10 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
docs/api/python/api_summary.html
View file

@ -236,7 +236,7 @@ facilitate the comparison.</p>
<dl class="py class">
<dt id="onnxruntime.InferenceSession">
<em class="property"><span class="pre">class</span> </em><code class="sig-prename descclassname"><span class="pre">onnxruntime.</span></code><code class="sig-name descname"><span class="pre">InferenceSession</span></code><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">path_or_bytes</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">sess_options</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">providers</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">provider_options</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="reference internal" href="modules/onnxruntime/capi/onnxruntime_inference_collection.html#InferenceSession"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#onnxruntime.InferenceSession" title="Permalink to this definition"></a></dt>
<dd><p>This is the main class used to run a model.</p>
<dd><p>This is the main class used to run a model. The next release (ORT 1.10) will require explicitly setting the providers parameter if you want to use execution providers other than the default CPU provider (as opposed to the current behavior of providers getting set/registered by default based on the build flags) when instantiating InferenceSession.</p>
</dd></dl>
<dl class="py class">

11
docs/api/python/auto_examples/plot_load_and_predict.html
View file

@ -53,9 +53,16 @@ retrieve the definition of its inputs and outputs.</p>
</pre></div>
</div>
<p>Let’s load a very simple model.
The model is available on github <a class="reference external" href="https://github.com/onnx/onnx/tree/master/onnx/backend/test/data/node/test_sigmoid">onnx…test_sigmoid</a>.</p>
The model is available on github <a class="reference external" href="https://github.com/onnx/onnx/tree/master/onnx/backend/test/data/node/test_sigmoid">onnx…test_sigmoid</a>.
Note: The next release (ORT 1.10) will require explicitly setting
the providers parameter if you want to use execution providers other
than the default CPU provider (as opposed to the current behavior of
providers getting set/registered by default based on the build flags) when
instantiating InferenceSession. Following code assumes NVIDIA GPU is available,
you can specify other execution providers or don't include providers parameter
to use default CPU provider.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">example1</span> <span class="o">=</span> <span class="n">get_example</span><span class="p">(</span><span class="s2">&quot;sigmoid.onnx&quot;</span><span class="p">)</span>
<span class="n">sess</span> <span class="o">=</span> <span class="n">rt</span><span class="o">.</span><span class="n">InferenceSession</span><span class="p">(</span><span class="n">example1</span><span class="p">)</span>
<span class="n">sess</span> <span class="o">=</span> <span class="n">rt</span><span class="o">.</span><span class="n">InferenceSession</span><span class="p">(</span><span class="n">example1</span><span class="p">,</span> <span class="n">providers=</span><span class="p">[</span><span class="s2">&quot;CUDAExecutionProvider&quot;</span><span class="p">]</span><span class="p">)</span>
</pre></div>
</div>
<p>Let’s see the input name and shape.</p>

5
docs/api/python/modules/onnxruntime/capi/onnxruntime_inference_collection.html
View file

@ -176,6 +176,11 @@
<span class="sd"> precedence. Values can either be provider names or tuples of</span>
<span class="sd"> (provider name, options dict). If not provided, then all available</span>
<span class="sd"> providers are used with the default precedence.</span>
<span class="sd"> The next release (ORT 1.10) will require explicitly setting</span>
<span class="sd"> this providers parameter if you want to use execution providers</span>
<span class="sd"> other than the default CPU provider (as opposed to the current</span>
<span class="sd"> behavior of providers getting set/registered by default based on the</span>
<span class="sd"> build flags) when instantiating InferenceSession.</span>
<span class="sd"> :param provider_options: Optional sequence of options dicts corresponding</span>
<span class="sd"> to the providers listed in &#39;providers&#39;.</span>

11
docs/api/python/tutorial.html
View file

@ -104,11 +104,18 @@ to convert other model formats into ONNX. Here we will use
<h2>Step 3: Load and run the model using ONNX Runtime<a class="headerlink" href="#step-3-load-and-run-the-model-using-onnx-runtime" title="Permalink to this headline"></a></h2>
<p>We will use <em>ONNX Runtime</em> to compute the predictions
for this machine learning model.</p>
<p>Note: The next release (ORT 1.10) will require explicitly setting
the providers parameter if you want to use execution providers other
than the default CPU provider (as opposed to the current behavior of
providers getting set/registered by default based on the build flags) when
instantiating InferenceSession. Following code assumes NVIDIA GPU is available,
you can specify other execution providers or don't include providers parameter
to use default CPU provider.</p>
<p>&lt;&lt;&lt;</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span>
<span class="kn">import</span> <span class="nn">onnxruntime</span> <span class="k">as</span> <span class="nn">rt</span>
<span class="n">sess</span> <span class="o">=</span> <span class="n">rt</span><span class="o">.</span><span class="n">InferenceSession</span><span class="p">(</span><span class="s2">&quot;logreg_iris.onnx&quot;</span><span class="p">)</span>
<span class="n">sess</span> <span class="o">=</span> <span class="n">rt</span><span class="o">.</span><span class="n">InferenceSession</span><span class="p">(</span><span class="s2">&quot;logreg_iris.onnx&quot;</span><span class="p">,</span> <span class="n">providers=</span><span class="p">[</span><span class="s2">&quot;CUDAExecutionProvider&quot;</span><span class="p">]</span><span class="p">)</span>
<span class="n">input_name</span> <span class="o">=</span> <span class="n">sess</span><span class="o">.</span><span class="n">get_inputs</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">name</span>
<span class="n">pred_onx</span> <span class="o">=</span> <span class="n">sess</span><span class="o">.</span><span class="n">run</span><span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="p">{</span><span class="n">input_name</span><span class="p">:</span> <span class="n">X_test</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="n">numpy</span><span class="o">.</span><span class="n">float32</span><span class="p">)})[</span><span class="mi">0</span><span class="p">]</span>
<span class="nb">print</span><span class="p">(</span><span class="n">pred_onx</span><span class="p">)</span>
@ -125,7 +132,7 @@ by specifying its name into a list.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span>
<span class="kn">import</span> <span class="nn">onnxruntime</span> <span class="k">as</span> <span class="nn">rt</span>
<span class="n">sess</span> <span class="o">=</span> <span class="n">rt</span><span class="o">.</span><span class="n">InferenceSession</span><span class="p">(</span><span class="s2">&quot;logreg_iris.onnx&quot;</span><span class="p">)</span>
<span class="n">sess</span> <span class="o">=</span> <span class="n">rt</span><span class="o">.</span><span class="n">InferenceSession</span><span class="p">(</span><span class="s2">&quot;logreg_iris.onnx&quot;</span><span class="p">,</span> <span class="n">providers=</span><span class="p">[</span><span class="s2">&quot;CUDAExecutionProvider&quot;</span><span class="p">]</span><span class="p">)</span>
<span class="n">input_name</span> <span class="o">=</span> <span class="n">sess</span><span class="o">.</span><span class="n">get_inputs</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">name</span>
<span class="n">label_name</span> <span class="o">=</span> <span class="n">sess</span><span class="o">.</span><span class="n">get_outputs</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">name</span>
<span class="n">pred_onx</span> <span class="o">=</span> <span class="n">sess</span><span class="o">.</span><span class="n">run</span><span class="p">(</span>

1
docs/execution-providers/MIGraphX-ExecutionProvider.md
View file

@ -37,6 +37,7 @@ The C API details are [here](../get-started/with-c.html.md).
When using the Python wheel from the ONNX Runtime build with MIGraphX execution provider, it will be automatically
prioritized over the default GPU or CPU execution providers. There is no need to separately register the execution
provider. Python APIs details are [here](/python/api_summary).
*Note that the next release (ORT 1.10) will require explicitly setting the providers parameter if you want to use execution provider other than the default CPU provider when instantiating InferenceSession.*
You can check [here](https://github.com/scxiao/ort_test/tree/master/python/run_onnx) for a python script to run an
model on either the CPU or MIGraphX Execution Provider.

7
docs/execution-providers/OpenVINO-ExecutionProvider.md
View file

@ -101,13 +101,12 @@ However, int8 support won't be available for VPU.
OpenVINO EP can be configured with certain options at runtime that control the behavior of the EP. These options can be set as key-value pairs as below:-
### Python API
Key-Value pairs for config options can be set using the Session.set_providers API as follows:-
Key-Value pairs for config options can be set using InferenceSession API as follow:-
```
session = onnxruntime.InferenceSession(<path_to_model_file>, options)
session.set_providers(['OpenVINOExecutionProvider'], [{Key1 : Value1, Key2 : Value2, ...}])
session = onnxruntime.InferenceSession(<path_to_model_file>, providers=['OpenVINOExecutionProvider'], provider_options=[{Key1 : Value1, Key2 : Value2, ...}])
```
*Note that this causes the InferenceSession to be re-initialized, which may cause model recompilation and hardware re-initialization*
*Note that the next release (ORT 1.10) will require explicitly setting the providers parameter if you want to use execution providers other than the default CPU provider (as opposed to the current behavior of providers getting set/registered by default based on the build flags) when instantiating InferenceSession.*
### C/C++ API
All the options shown below are passed to SessionOptionsAppendExecutionProvider_OpenVINO() API and populated in the struct OrtOpenVINOProviderOptions in an example shown below, for example for CPU device type:

1
docs/execution-providers/TensorRT-ExecutionProvider.md
View file

@ -46,6 +46,7 @@ If some operators in the model are not supported by TensorRT, ONNX Runtime will
### Python
When using the Python wheel from the ONNX Runtime build with TensorRT execution provider, it will be automatically prioritized over the default GPU or CPU execution providers. There is no need to separately register the execution provider.
*Note that the next release (ORT 1.10) will require explicitly setting the providers parameter if you want to use execution providers other than the default CPU provider when instantiating InferenceSession.*
## Configurations

8
docs/tutorials/tf-get-started.md
View file

@ -78,7 +78,13 @@ import numpy as np
# Change shapes and types to match model
input1 = np.zeros((1, 100, 100, 3), np.float32)
sess = ort.InferenceSession("dst/path/model.onnx")
# Start from ORT 1.10, ORT requires explicitly setting the providers parameter if you want to use execution providers
# other than the default CPU provider (as opposed to the previous behavior of providers getting set/registered by default
# based on the build flags) when instantiating InferenceSession.
# Following code assumes NVIDIA GPU is available, you can specify other execution providers or don't include providers parameter
# to use default CPU provider.
sess = ort.InferenceSession("dst/path/model.onnx", providers=["CUDAExecutionProvider"])
# Set first argument of sess.run to None to use all model outputs in default order
# Input/output names are printed by the CLI and can be set with --rename-inputs and --rename-outputs
# If using the python API, names are determined from function arg names or TensorSpec names.