mirror of
https://github.com/saymrwulf/onnxruntime.git
synced 2026-07-14 18:12:05 +00:00
a0f3e30de6
* initial setup and rename "how to" to "setup" * move API to main nav * move api to main nav * add get starated, rework nav order * rename to install move mds out of install section * update api nav and home page * add install docs and python qs updates * python get started work * remove c and obj c for now * move java, python, and obj-c docs under api folder * move java api html to iframe (ugh) * remove api docs w/o details, move api text getstar * remove api docs wo detail updates get started * remvoe iframes * move eco system to main nav * fix api buttons * added more examples moved intro to ORT * fix links * fix get started titles * fix get started titles * fix more links * fix more links * more link fixes * fix nav remove inferencing and training subnav * fix top nav remove inference and training nav * fix title * fix tutorials nav hierarchy * fix python api button * add tenorflow keras example * fix quickstart toc * add imports fix spacing * fix links * update nav and python get started page * move ort training example, add coming soon for iot * update C# get started * fix spacing on quantization * Add some js get started content * fix formatting * fix typo * removed onnx-pytorch and onnx-tf * updated pip install torch and added links iot page * added pytorch tutorial heirarchy * updated web to docs soon added release blog link * add web link
216 lines
No EOL
14 KiB
HTML
216 lines
No EOL
14 KiB
HTML
|
||
<!DOCTYPE html>
|
||
|
||
<html lang="en">
|
||
<head>
|
||
<meta charset="utf-8" />
|
||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||
<title>Profile the execution of a simple model — ONNX Runtime 1.7.0 documentation</title>
|
||
<link rel="stylesheet" href="../static/pygments.css" type="text/css" />
|
||
<link rel="stylesheet" href="../static/alabaster.css" type="text/css" />
|
||
<link rel="stylesheet" type="text/css" href="../static/graphviz.css" />
|
||
<link rel="stylesheet" type="text/css" href="../static/gallery.css" />
|
||
<link rel="stylesheet" type="text/css" href="../static/gallery-binder.css" />
|
||
<link rel="stylesheet" type="text/css" href="../static/gallery-dataframe.css" />
|
||
<link rel="stylesheet" type="text/css" href="../static/gallery-rendered-html.css" />
|
||
<script id="documentation_options" data-url_root="../" src="../static/documentation_options.js"></script>
|
||
<script src="../static/jquery.js"></script>
|
||
<script src="../static/underscore.js"></script>
|
||
<script src="../static/doctools.js"></script>
|
||
<link rel="index" title="Index" href="../genindex.html" />
|
||
<link rel="search" title="Search" href="../search.html" />
|
||
<link rel="next" title="Train, convert and predict with ONNX Runtime" href="plot_convert_pipeline_vectorizer.html" />
|
||
<link rel="prev" title="Metadata" href="plot_metadata.html" />
|
||
|
||
<link rel="stylesheet" href="../static/custom.css" type="text/css" />
|
||
|
||
|
||
<meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
|
||
|
||
</head><body>
|
||
|
||
|
||
<div class="document">
|
||
<div class="documentwrapper">
|
||
<div class="bodywrapper">
|
||
|
||
|
||
<div class="body" role="main">
|
||
|
||
<div class="sphx-glr-download-link-note admonition note">
|
||
<p class="admonition-title">Note</p>
|
||
<p>Click <a class="reference internal" href="#sphx-glr-download-auto-examples-plot-profiling-py"><span class="std std-ref">here</span></a>
|
||
to download the full example code</p>
|
||
</div>
|
||
<div class="sphx-glr-example-title section" id="profile-the-execution-of-a-simple-model">
|
||
<span id="l-example-profiling"></span><span id="sphx-glr-auto-examples-plot-profiling-py"></span><h1>Profile the execution of a simple model<a class="headerlink" href="#profile-the-execution-of-a-simple-model" title="Permalink to this headline">¶</a></h1>
|
||
<p><em>ONNX Runtime</em> can profile the execution of the model.
|
||
This example shows how to interpret the results.</p>
|
||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">onnx</span>
|
||
<span class="kn">import</span> <span class="nn">onnxruntime</span> <span class="k">as</span> <span class="nn">rt</span>
|
||
<span class="kn">import</span> <span class="nn">numpy</span>
|
||
<span class="kn">from</span> <span class="nn">onnxruntime.datasets</span> <span class="kn">import</span> <span class="n">get_example</span>
|
||
|
||
|
||
<span class="k">def</span> <span class="nf">change_ir_version</span><span class="p">(</span><span class="n">filename</span><span class="p">,</span> <span class="n">ir_version</span><span class="o">=</span><span class="mi">6</span><span class="p">):</span>
|
||
<span class="s2">"onnxruntime==1.2.0 does not support opset <= 7 and ir_version > 6"</span>
|
||
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">filename</span><span class="p">,</span> <span class="s2">"rb"</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
|
||
<span class="n">model</span> <span class="o">=</span> <span class="n">onnx</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
|
||
<span class="n">model</span><span class="o">.</span><span class="n">ir_version</span> <span class="o">=</span> <span class="mi">6</span>
|
||
<span class="k">if</span> <span class="n">model</span><span class="o">.</span><span class="n">opset_import</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">version</span> <span class="o"><=</span> <span class="mi">7</span><span class="p">:</span>
|
||
<span class="n">model</span><span class="o">.</span><span class="n">opset_import</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">version</span> <span class="o">=</span> <span class="mi">11</span>
|
||
<span class="k">return</span> <span class="n">model</span>
|
||
</pre></div>
|
||
</div>
|
||
<p>Let’s load a very simple model and compute some prediction.</p>
|
||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">example1</span> <span class="o">=</span> <span class="n">get_example</span><span class="p">(</span><span class="s2">"mul_1.onnx"</span><span class="p">)</span>
|
||
<span class="n">onnx_model</span> <span class="o">=</span> <span class="n">change_ir_version</span><span class="p">(</span><span class="n">example1</span><span class="p">)</span>
|
||
<span class="n">onnx_model_str</span> <span class="o">=</span> <span class="n">onnx_model</span><span class="o">.</span><span class="n">SerializeToString</span><span class="p">()</span>
|
||
<span class="n">sess</span> <span class="o">=</span> <span class="n">rt</span><span class="o">.</span><span class="n">InferenceSession</span><span class="p">(</span><span class="n">onnx_model_str</span><span class="p">)</span>
|
||
<span class="n">input_name</span> <span class="o">=</span> <span class="n">sess</span><span class="o">.</span><span class="n">get_inputs</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">name</span>
|
||
|
||
<span class="n">x</span> <span class="o">=</span> <span class="n">numpy</span><span class="o">.</span><span class="n">array</span><span class="p">([[</span><span class="mf">1.0</span><span class="p">,</span> <span class="mf">2.0</span><span class="p">],</span> <span class="p">[</span><span class="mf">3.0</span><span class="p">,</span> <span class="mf">4.0</span><span class="p">],</span> <span class="p">[</span><span class="mf">5.0</span><span class="p">,</span> <span class="mf">6.0</span><span class="p">]],</span> <span class="n">dtype</span><span class="o">=</span><span class="n">numpy</span><span class="o">.</span><span class="n">float32</span><span class="p">)</span>
|
||
<span class="n">res</span> <span class="o">=</span> <span class="n">sess</span><span class="o">.</span><span class="n">run</span><span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="p">{</span><span class="n">input_name</span><span class="p">:</span> <span class="n">x</span><span class="p">})</span>
|
||
<span class="nb">print</span><span class="p">(</span><span class="n">res</span><span class="p">)</span>
|
||
</pre></div>
|
||
</div>
|
||
<p class="sphx-glr-script-out">Out:</p>
|
||
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>[array([[ 1., 4.],
|
||
[ 9., 16.],
|
||
[25., 36.]], dtype=float32)]
|
||
</pre></div>
|
||
</div>
|
||
<p>We need to enable to profiling
|
||
before running the predictions.</p>
|
||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">options</span> <span class="o">=</span> <span class="n">rt</span><span class="o">.</span><span class="n">SessionOptions</span><span class="p">()</span>
|
||
<span class="n">options</span><span class="o">.</span><span class="n">enable_profiling</span> <span class="o">=</span> <span class="kc">True</span>
|
||
<span class="n">sess_profile</span> <span class="o">=</span> <span class="n">rt</span><span class="o">.</span><span class="n">InferenceSession</span><span class="p">(</span><span class="n">onnx_model_str</span><span class="p">,</span> <span class="n">options</span><span class="p">)</span>
|
||
<span class="n">input_name</span> <span class="o">=</span> <span class="n">sess</span><span class="o">.</span><span class="n">get_inputs</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">name</span>
|
||
|
||
<span class="n">x</span> <span class="o">=</span> <span class="n">numpy</span><span class="o">.</span><span class="n">array</span><span class="p">([[</span><span class="mf">1.0</span><span class="p">,</span> <span class="mf">2.0</span><span class="p">],</span> <span class="p">[</span><span class="mf">3.0</span><span class="p">,</span> <span class="mf">4.0</span><span class="p">],</span> <span class="p">[</span><span class="mf">5.0</span><span class="p">,</span> <span class="mf">6.0</span><span class="p">]],</span> <span class="n">dtype</span><span class="o">=</span><span class="n">numpy</span><span class="o">.</span><span class="n">float32</span><span class="p">)</span>
|
||
|
||
<span class="n">sess</span><span class="o">.</span><span class="n">run</span><span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="p">{</span><span class="n">input_name</span><span class="p">:</span> <span class="n">x</span><span class="p">})</span>
|
||
<span class="n">prof_file</span> <span class="o">=</span> <span class="n">sess_profile</span><span class="o">.</span><span class="n">end_profiling</span><span class="p">()</span>
|
||
<span class="nb">print</span><span class="p">(</span><span class="n">prof_file</span><span class="p">)</span>
|
||
</pre></div>
|
||
</div>
|
||
<p class="sphx-glr-script-out">Out:</p>
|
||
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>onnxruntime_profile__2021-03-04_18-40-00.json
|
||
</pre></div>
|
||
</div>
|
||
<p>The results are stored un a file in JSON format.
|
||
Let’s see what it contains.</p>
|
||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">json</span>
|
||
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">prof_file</span><span class="p">,</span> <span class="s2">"r"</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
|
||
<span class="n">sess_time</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
|
||
<span class="kn">import</span> <span class="nn">pprint</span>
|
||
<span class="n">pprint</span><span class="o">.</span><span class="n">pprint</span><span class="p">(</span><span class="n">sess_time</span><span class="p">)</span>
|
||
</pre></div>
|
||
</div>
|
||
<p class="sphx-glr-script-out">Out:</p>
|
||
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>[{'args': {},
|
||
'cat': 'Session',
|
||
'dur': 101,
|
||
'name': 'model_loading_array',
|
||
'ph': 'X',
|
||
'pid': 77,
|
||
'tid': 77,
|
||
'ts': 3},
|
||
{'args': {},
|
||
'cat': 'Session',
|
||
'dur': 227,
|
||
'name': 'session_initialization',
|
||
'ph': 'X',
|
||
'pid': 77,
|
||
'tid': 77,
|
||
'ts': 134}]
|
||
</pre></div>
|
||
</div>
|
||
<p class="sphx-glr-timing"><strong>Total running time of the script:</strong> ( 0 minutes 0.010 seconds)</p>
|
||
<div class="sphx-glr-footer class sphx-glr-footer-example docutils container" id="sphx-glr-download-auto-examples-plot-profiling-py">
|
||
<div class="sphx-glr-download sphx-glr-download-python docutils container">
|
||
<p><a class="reference download internal" download="" href="../downloads/cfe61aca1f0a89486c7024466ea500fd/plot_profiling.py"><code class="xref download docutils literal notranslate"><span class="pre">Download</span> <span class="pre">Python</span> <span class="pre">source</span> <span class="pre">code:</span> <span class="pre">plot_profiling.py</span></code></a></p>
|
||
</div>
|
||
<div class="sphx-glr-download sphx-glr-download-jupyter docutils container">
|
||
<p><a class="reference download internal" download="" href="../downloads/ccbbfd4f60683438ab99a4c42d3fbbdf/plot_profiling.ipynb"><code class="xref download docutils literal notranslate"><span class="pre">Download</span> <span class="pre">Jupyter</span> <span class="pre">notebook:</span> <span class="pre">plot_profiling.ipynb</span></code></a></p>
|
||
</div>
|
||
</div>
|
||
<p class="sphx-glr-signature"><a class="reference external" href="https://sphinx-gallery.github.io">Gallery generated by Sphinx-Gallery</a></p>
|
||
</div>
|
||
|
||
|
||
</div>
|
||
|
||
</div>
|
||
</div>
|
||
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
|
||
<div class="sphinxsidebarwrapper">
|
||
<p class="logo"><a href="../index.html">
|
||
<img class="logo" src="../static/ONNX_Runtime_icon.png" alt="Logo"/>
|
||
</a></p>
|
||
<h1 class="logo"><a href="../index.html">ONNX Runtime</a></h1>
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
<h3>Navigation</h3>
|
||
<ul>
|
||
<li class="toctree-l1"><a class="reference internal" href="../tutorial.html">Tutorial</a></li>
|
||
<li class="toctree-l1"><a class="reference internal" href="../api_summary.html">API Summary</a></li>
|
||
<li class="toctree-l1"><a class="reference internal" href="index.html">Gallery of examples</a></li>
|
||
</ul>
|
||
|
||
<div class="relations">
|
||
<h3>Related Topics</h3>
|
||
<ul>
|
||
<li><a href="../index.html">Documentation overview</a><ul>
|
||
<li><a href="index.html">Gallery of examples</a><ul>
|
||
<li>Previous: <a href="plot_metadata.html" title="previous chapter">Metadata</a></li>
|
||
<li>Next: <a href="plot_convert_pipeline_vectorizer.html" title="next chapter">Train, convert and predict with ONNX Runtime</a></li>
|
||
</ul></li>
|
||
</ul></li>
|
||
</ul>
|
||
</div>
|
||
<div id="searchbox" style="display: none" role="search">
|
||
<h3 id="searchlabel">Quick search</h3>
|
||
<div class="searchformwrapper">
|
||
<form class="search" action="../search.html" method="get">
|
||
<input type="text" name="q" aria-labelledby="searchlabel" />
|
||
<input type="submit" value="Go" />
|
||
</form>
|
||
</div>
|
||
</div>
|
||
<script>$('#searchbox').show(0);</script>
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
</div>
|
||
</div>
|
||
<div class="clearer"></div>
|
||
</div>
|
||
<div class="footer">
|
||
©2018-2021, Microsoft.
|
||
|
||
|
|
||
Powered by <a href="http://sphinx-doc.org/">Sphinx 3.5.1</a>
|
||
& <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.12</a>
|
||
|
||
|
|
||
<a href="../sources/auto_examples/plot_profiling.rst.txt"
|
||
rel="nofollow">Page source</a>
|
||
</div>
|
||
|
||
|
||
|
||
|
||
</body>
|
||
</html> |