mirror of
https://github.com/saymrwulf/onnxruntime.git
synced 2026-07-14 18:12:05 +00:00
194 lines
No EOL
19 KiB
HTML
194 lines
No EOL
19 KiB
HTML
<!DOCTYPE html>
|
||
<html lang="en">
|
||
<head>
|
||
<meta charset="utf-8">
|
||
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
|
||
<meta http-equiv="x-ua-compatible" content="ie=edge">
|
||
|
||
<title>Tutorial</title>
|
||
|
||
<link rel="stylesheet" href="_static/sphinx-modern-theme.css" type="text/css"/>
|
||
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
|
||
<link rel="stylesheet" href="_static/gallery.css" type="text/css" />
|
||
</head>
|
||
<body>
|
||
<div class="container">
|
||
<div class="row" style="margin-top: 1rem;">
|
||
<div id="sidebar" class="col-xs-12 col-sm-3">
|
||
<a href="index.html">
|
||
<img style="margin-bottom: 0.5rem;" class="img-fluid" src="_static/ONNX_Runtime_icon.png"/>
|
||
</a>
|
||
<div id="searchbox" style="display: none" role="search">
|
||
<form class="form-inline" action="search.html" method="get">
|
||
<div class="form-group">
|
||
<label class="sr-only" for="searchInput">Search</label>
|
||
<input type="text" class="form-control" name="q" id="searchInput" placeholder="Search">
|
||
</div>
|
||
<button type="submit" class="btn btn-secondary" style="display:none">Go</button>
|
||
<input type="hidden" name="check_keywords" value="yes"/>
|
||
<input type="hidden" name="area" value="default"/>
|
||
</form>
|
||
</div>
|
||
|
||
<hr>
|
||
|
||
<div id="toc">
|
||
<ul class="current">
|
||
<li class="toctree-l1 current"><a class="current reference internal" href="#">Tutorial</a><ul>
|
||
<li class="toctree-l2"><a class="reference internal" href="#step-1-train-a-model-using-your-favorite-framework">Step 1: Train a model using your favorite framework</a></li>
|
||
<li class="toctree-l2"><a class="reference internal" href="#step-2-convert-or-export-the-model-into-onnx-format">Step 2: Convert or export the model into ONNX format</a></li>
|
||
<li class="toctree-l2"><a class="reference internal" href="#step-3-load-and-run-the-model-using-onnx-runtime">Step 3: Load and run the model using ONNX Runtime</a></li>
|
||
</ul>
|
||
</li>
|
||
<li class="toctree-l1"><a class="reference internal" href="api_summary.html">API Summary</a><ul>
|
||
<li class="toctree-l2"><a class="reference internal" href="api_summary.html#device">Device</a></li>
|
||
<li class="toctree-l2"><a class="reference internal" href="api_summary.html#examples-and-datasets">Examples and datasets</a></li>
|
||
<li class="toctree-l2"><a class="reference internal" href="api_summary.html#load-and-run-a-model">Load and run a model</a></li>
|
||
<li class="toctree-l2"><a class="reference internal" href="api_summary.html#backend">Backend</a></li>
|
||
</ul>
|
||
</li>
|
||
<li class="toctree-l1"><a class="reference internal" href="auto_examples/index.html">Gallery of examples</a><ul>
|
||
<li class="toctree-l2"><a class="reference internal" href="auto_examples/plot_backend.html">ONNX Runtime Backend for ONNX</a></li>
|
||
<li class="toctree-l2"><a class="reference internal" href="auto_examples/plot_pipeline.html">Draw a pipeline</a></li>
|
||
<li class="toctree-l2"><a class="reference internal" href="auto_examples/plot_load_and_predict.html">Load and predict with ONNX Runtime and a very simple model</a></li>
|
||
<li class="toctree-l2"><a class="reference internal" href="auto_examples/plot_profiling.html">Profile the execution of a simple model</a></li>
|
||
<li class="toctree-l2"><a class="reference internal" href="auto_examples/plot_metadata.html">Metadata</a></li>
|
||
<li class="toctree-l2"><a class="reference internal" href="auto_examples/plot_dl_keras.html">ONNX Runtime for Keras</a></li>
|
||
<li class="toctree-l2"><a class="reference internal" href="auto_examples/plot_convert_pipeline_vectorizer.html">Train, convert and predict with ONNX Runtime</a></li>
|
||
<li class="toctree-l2"><a class="reference internal" href="auto_examples/plot_common_errors.html">Common errors with onnxruntime</a></li>
|
||
<li class="toctree-l2"><a class="reference internal" href="auto_examples/plot_train_convert_predict.html">Train, convert and predict with ONNX Runtime</a></li>
|
||
</ul>
|
||
</li>
|
||
</ul>
|
||
|
||
</div>
|
||
</div>
|
||
<div class="col-xs-12 col-sm-9">
|
||
<div class="section" id="tutorial">
|
||
<h1>Tutorial<a class="headerlink" href="#tutorial" title="Permalink to this headline">¶</a></h1>
|
||
<p><em>ONNX Runtime</em> provides an easy way to run
|
||
machine learned models with high performance on CPU or GPU
|
||
without dependencies on the training framework.
|
||
Machine learning frameworks are usually optimized for
|
||
batch training rather than for prediction, which is a
|
||
more common scenario in applications, sites, and services.
|
||
At a high level, you can:</p>
|
||
<ol class="arabic simple">
|
||
<li><p>Train a model using your favorite framework.</p></li>
|
||
<li><p>Convert or export the model into ONNX format.
|
||
See <a class="reference external" href="https://github.com/onnx/tutorials">ONNX Tutorials</a>
|
||
for more details.</p></li>
|
||
<li><p>Load and run the model using <em>ONNX Runtime</em>.</p></li>
|
||
</ol>
|
||
<p>In this tutorial, we will briefly create a
|
||
pipeline with <em>scikit-learn</em>, convert it into
|
||
ONNX format and run the first predictions.</p>
|
||
<div class="section" id="step-1-train-a-model-using-your-favorite-framework">
|
||
<h2>Step 1: Train a model using your favorite framework<a class="headerlink" href="#step-1-train-a-model-using-your-favorite-framework" title="Permalink to this headline">¶</a></h2>
|
||
<p>We’ll use the famous iris datasets.</p>
|
||
<p><<<</p>
|
||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">sklearn.datasets</span> <span class="k">import</span> <span class="n">load_iris</span>
|
||
<span class="kn">from</span> <span class="nn">sklearn.model_selection</span> <span class="k">import</span> <span class="n">train_test_split</span>
|
||
<span class="n">iris</span> <span class="o">=</span> <span class="n">load_iris</span><span class="p">()</span>
|
||
<span class="n">X</span><span class="p">,</span> <span class="n">y</span> <span class="o">=</span> <span class="n">iris</span><span class="o">.</span><span class="n">data</span><span class="p">,</span> <span class="n">iris</span><span class="o">.</span><span class="n">target</span>
|
||
<span class="n">X_train</span><span class="p">,</span> <span class="n">X_test</span><span class="p">,</span> <span class="n">y_train</span><span class="p">,</span> <span class="n">y_test</span> <span class="o">=</span> <span class="n">train_test_split</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">y</span><span class="p">)</span>
|
||
|
||
<span class="kn">from</span> <span class="nn">sklearn.linear_model</span> <span class="k">import</span> <span class="n">LogisticRegression</span>
|
||
<span class="n">clr</span> <span class="o">=</span> <span class="n">LogisticRegression</span><span class="p">()</span>
|
||
<span class="n">clr</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">X_train</span><span class="p">,</span> <span class="n">y_train</span><span class="p">)</span>
|
||
<span class="nb">print</span><span class="p">(</span><span class="n">clr</span><span class="p">)</span>
|
||
</pre></div>
|
||
</div>
|
||
<p>>>></p>
|
||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span> <span class="n">LogisticRegression</span><span class="p">(</span><span class="n">C</span><span class="o">=</span><span class="mf">1.0</span><span class="p">,</span> <span class="n">class_weight</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">dual</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">fit_intercept</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span>
|
||
<span class="n">intercept_scaling</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">l1_ratio</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">max_iter</span><span class="o">=</span><span class="mi">100</span><span class="p">,</span>
|
||
<span class="n">multi_class</span><span class="o">=</span><span class="s1">'warn'</span><span class="p">,</span> <span class="n">n_jobs</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">penalty</span><span class="o">=</span><span class="s1">'l2'</span><span class="p">,</span>
|
||
<span class="n">random_state</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">solver</span><span class="o">=</span><span class="s1">'warn'</span><span class="p">,</span> <span class="n">tol</span><span class="o">=</span><span class="mf">0.0001</span><span class="p">,</span> <span class="n">verbose</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span>
|
||
<span class="n">warm_start</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
|
||
</pre></div>
|
||
</div>
|
||
</div>
|
||
<div class="section" id="step-2-convert-or-export-the-model-into-onnx-format">
|
||
<h2>Step 2: Convert or export the model into ONNX format<a class="headerlink" href="#step-2-convert-or-export-the-model-into-onnx-format" title="Permalink to this headline">¶</a></h2>
|
||
<p><a class="reference external" href="https://github.com/onnx/onnx">ONNX</a> is a format to describe
|
||
the machine learned model.
|
||
It defines a set of commonly used operators to compose models.
|
||
There are <a class="reference external" href="https://github.com/onnx/tutorials">tools</a>
|
||
to convert other model formats into ONNX. Here we will use
|
||
<a class="reference external" href="https://github.com/onnx/onnxmltools">ONNXMLTools</a>.</p>
|
||
<p><<<</p>
|
||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">skl2onnx</span> <span class="k">import</span> <span class="n">convert_sklearn</span>
|
||
<span class="kn">from</span> <span class="nn">skl2onnx.common.data_types</span> <span class="k">import</span> <span class="n">FloatTensorType</span>
|
||
|
||
<span class="n">initial_type</span> <span class="o">=</span> <span class="p">[(</span><span class="s1">'float_input'</span><span class="p">,</span> <span class="n">FloatTensorType</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">4</span><span class="p">]))]</span>
|
||
<span class="n">onx</span> <span class="o">=</span> <span class="n">convert_sklearn</span><span class="p">(</span><span class="n">clr</span><span class="p">,</span> <span class="n">initial_types</span><span class="o">=</span><span class="n">initial_type</span><span class="p">)</span>
|
||
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s2">"logreg_iris.onnx"</span><span class="p">,</span> <span class="s2">"wb"</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
|
||
<span class="n">f</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">onx</span><span class="o">.</span><span class="n">SerializeToString</span><span class="p">())</span>
|
||
</pre></div>
|
||
</div>
|
||
<p>>>></p>
|
||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span> <span class="n">The</span> <span class="n">maximum</span> <span class="n">opset</span> <span class="n">needed</span> <span class="n">by</span> <span class="n">this</span> <span class="n">model</span> <span class="ow">is</span> <span class="n">only</span> <span class="mf">9.</span>
|
||
<span class="n">The</span> <span class="n">maximum</span> <span class="n">opset</span> <span class="n">needed</span> <span class="n">by</span> <span class="n">this</span> <span class="n">model</span> <span class="ow">is</span> <span class="n">only</span> <span class="mf">1.</span>
|
||
</pre></div>
|
||
</div>
|
||
</div>
|
||
<div class="section" id="step-3-load-and-run-the-model-using-onnx-runtime">
|
||
<h2>Step 3: Load and run the model using ONNX Runtime<a class="headerlink" href="#step-3-load-and-run-the-model-using-onnx-runtime" title="Permalink to this headline">¶</a></h2>
|
||
<p>We will use <em>ONNX Runtime</em> to compute the predictions
|
||
for this machine learning model.</p>
|
||
<p><<<</p>
|
||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span>
|
||
<span class="kn">import</span> <span class="nn">onnxruntime</span> <span class="k">as</span> <span class="nn">rt</span>
|
||
|
||
<span class="n">sess</span> <span class="o">=</span> <span class="n">rt</span><span class="o">.</span><span class="n">InferenceSession</span><span class="p">(</span><span class="s2">"logreg_iris.onnx"</span><span class="p">)</span>
|
||
<span class="n">input_name</span> <span class="o">=</span> <span class="n">sess</span><span class="o">.</span><span class="n">get_inputs</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">name</span>
|
||
<span class="n">pred_onx</span> <span class="o">=</span> <span class="n">sess</span><span class="o">.</span><span class="n">run</span><span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="p">{</span><span class="n">input_name</span><span class="p">:</span> <span class="n">X_test</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="n">numpy</span><span class="o">.</span><span class="n">float32</span><span class="p">)})[</span><span class="mi">0</span><span class="p">]</span>
|
||
<span class="nb">print</span><span class="p">(</span><span class="n">pred_onx</span><span class="p">)</span>
|
||
</pre></div>
|
||
</div>
|
||
<p>>>></p>
|
||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span> <span class="p">[</span><span class="mi">1</span> <span class="mi">0</span> <span class="mi">2</span> <span class="mi">2</span> <span class="mi">2</span> <span class="mi">0</span> <span class="mi">2</span> <span class="mi">2</span> <span class="mi">2</span> <span class="mi">2</span> <span class="mi">2</span> <span class="mi">0</span> <span class="mi">1</span> <span class="mi">1</span> <span class="mi">1</span> <span class="mi">0</span> <span class="mi">1</span> <span class="mi">1</span> <span class="mi">2</span> <span class="mi">1</span> <span class="mi">2</span> <span class="mi">2</span> <span class="mi">2</span> <span class="mi">1</span> <span class="mi">1</span> <span class="mi">0</span> <span class="mi">0</span> <span class="mi">0</span> <span class="mi">2</span> <span class="mi">0</span> <span class="mi">2</span> <span class="mi">2</span> <span class="mi">2</span> <span class="mi">2</span> <span class="mi">0</span> <span class="mi">1</span> <span class="mi">2</span>
|
||
<span class="mi">2</span><span class="p">]</span>
|
||
</pre></div>
|
||
</div>
|
||
<p>The code can be changed to get one specific output
|
||
by specifying its name into a list.</p>
|
||
<p><<<</p>
|
||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span>
|
||
<span class="kn">import</span> <span class="nn">onnxruntime</span> <span class="k">as</span> <span class="nn">rt</span>
|
||
|
||
<span class="n">sess</span> <span class="o">=</span> <span class="n">rt</span><span class="o">.</span><span class="n">InferenceSession</span><span class="p">(</span><span class="s2">"logreg_iris.onnx"</span><span class="p">)</span>
|
||
<span class="n">input_name</span> <span class="o">=</span> <span class="n">sess</span><span class="o">.</span><span class="n">get_inputs</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">name</span>
|
||
<span class="n">label_name</span> <span class="o">=</span> <span class="n">sess</span><span class="o">.</span><span class="n">get_outputs</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">name</span>
|
||
<span class="n">pred_onx</span> <span class="o">=</span> <span class="n">sess</span><span class="o">.</span><span class="n">run</span><span class="p">(</span>
|
||
<span class="p">[</span><span class="n">label_name</span><span class="p">],</span> <span class="p">{</span><span class="n">input_name</span><span class="p">:</span> <span class="n">X_test</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="n">numpy</span><span class="o">.</span><span class="n">float32</span><span class="p">)})[</span><span class="mi">0</span><span class="p">]</span>
|
||
<span class="nb">print</span><span class="p">(</span><span class="n">pred_onx</span><span class="p">)</span>
|
||
</pre></div>
|
||
</div>
|
||
<p>>>></p>
|
||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span> <span class="p">[</span><span class="mi">1</span> <span class="mi">0</span> <span class="mi">2</span> <span class="mi">2</span> <span class="mi">2</span> <span class="mi">0</span> <span class="mi">2</span> <span class="mi">2</span> <span class="mi">2</span> <span class="mi">2</span> <span class="mi">2</span> <span class="mi">0</span> <span class="mi">1</span> <span class="mi">1</span> <span class="mi">1</span> <span class="mi">0</span> <span class="mi">1</span> <span class="mi">1</span> <span class="mi">2</span> <span class="mi">1</span> <span class="mi">2</span> <span class="mi">2</span> <span class="mi">2</span> <span class="mi">1</span> <span class="mi">1</span> <span class="mi">0</span> <span class="mi">0</span> <span class="mi">0</span> <span class="mi">2</span> <span class="mi">0</span> <span class="mi">2</span> <span class="mi">2</span> <span class="mi">2</span> <span class="mi">2</span> <span class="mi">0</span> <span class="mi">1</span> <span class="mi">2</span>
|
||
<span class="mi">2</span><span class="p">]</span>
|
||
</pre></div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
|
||
</div>
|
||
</div>
|
||
</div>
|
||
|
||
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.0.0/jquery.min.js"
|
||
integrity="sha384-THPy051/pYDQGanwU6poAc/hOdQxjnOEXzbT+OuUAFqNqFjL+4IGLBgCJC3ZOShY"
|
||
crossorigin="anonymous"></script>
|
||
<script src="https://cdnjs.cloudflare.com/ajax/libs/tether/1.2.0/js/tether.min.js"
|
||
integrity="sha384-Plbmg8JY28KFelvJVai01l8WyZzrYWG825m+cZ0eDDS1f7d/js6ikvy1+X+guPIB"
|
||
crossorigin="anonymous"></script>
|
||
<script src="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0-alpha.3/js/bootstrap.min.js"
|
||
integrity="sha384-ux8v3A6CPtOTqOzMKiuo3d/DomGaaClxFYdCu2HPMBEkf6x2xiDyJ7gkXU0MWwaD"
|
||
crossorigin="anonymous"></script>
|
||
<script src='https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML'></script>
|
||
<script src="https://cdnjs.cloudflare.com/ajax/libs/lunr.js/0.6.0/lunr.min.js"></script>
|
||
<script src="_static/searchtools.js"></script>
|
||
<script>$('#searchbox').show(0)</script>
|
||
</body>
|
||
</html> |