saymrwulf/onnxruntime
1
0
Fork 0
mirror of https://github.com/saymrwulf/onnxruntime.git synced 2026-07-14 18:12:05 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
onnxruntime/docs/api/python/auto_examples/plot_train_convert_predict.html
github-actions[bot] 8bde251409
Update Python API docs to commit 9765949 (#10181) 
Co-authored-by: natke <natke@users.noreply.github.com>
2022-01-04 09:32:40 -08:00

440 lines
No EOL
39 KiB
HTML
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="generator" content="Docutils 0.17.1: http://docutils.sourceforge.net/" />
<title>Train, convert and predict with ONNX Runtime &#8212; ONNX Runtime 1.11.0 documentation</title>
<link rel="stylesheet" type="text/css" href="../static/pygments.css" />
<link rel="stylesheet" type="text/css" href="../static/alabaster.css" />
<link rel="stylesheet" type="text/css" href="../static/graphviz.css" />
<link rel="stylesheet" type="text/css" href="../static/sg_gallery.css" />
<link rel="stylesheet" type="text/css" href="../static/sg_gallery-binder.css" />
<link rel="stylesheet" type="text/css" href="../static/sg_gallery-dataframe.css" />
<link rel="stylesheet" type="text/css" href="../static/sg_gallery-rendered-html.css" />
<script data-url_root="../" id="documentation_options" src="../static/documentation_options.js"></script>
<script src="../static/jquery.js"></script>
<script src="../static/underscore.js"></script>
<script src="../static/doctools.js"></script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="prev" title="Common errors with onnxruntime" href="plot_common_errors.html" />
<link rel="stylesheet" href="../static/custom.css" type="text/css" />
<meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
</head><body>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<div class="sphx-glr-download-link-note admonition note">
<p class="admonition-title">Note</p>
<p>Click <a class="reference internal" href="#sphx-glr-download-auto-examples-plot-train-convert-predict-py"><span class="std std-ref">here</span></a>
to download the full example code</p>
</div>
<section class="sphx-glr-example-title" id="train-convert-and-predict-with-onnx-runtime">
<span id="l-logreg-example"></span><span id="sphx-glr-auto-examples-plot-train-convert-predict-py"></span><h1>Train, convert and predict with ONNX Runtime<a class="headerlink" href="#train-convert-and-predict-with-onnx-runtime" title="Permalink to this headline"></a></h1>
<p>This example demonstrates an end to end scenario
starting with the training of a machine learned model
to its use in its converted from.</p>
<div class="contents local topic" id="contents">
<ul class="simple">
<li><p><a class="reference internal" href="#train-a-logistic-regression" id="id1">Train a logistic regression</a></p></li>
<li><p><a class="reference internal" href="#conversion-to-onnx-format" id="id2">Conversion to ONNX format</a></p></li>
<li><p><a class="reference internal" href="#probabilities" id="id3">Probabilities</a></p></li>
<li><p><a class="reference internal" href="#benchmark-with-randomforest" id="id4">Benchmark with RandomForest</a></p></li>
</ul>
</div>
<section id="train-a-logistic-regression">
<h2><a class="toc-backref" href="#id1">Train a logistic regression</a><a class="headerlink" href="#train-a-logistic-regression" title="Permalink to this headline"></a></h2>
<p>The first step consists in retrieving the iris datset.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">sklearn.datasets</span> <span class="k">import</span> <span class="n">load_iris</span>
<span class="n">iris</span> <span class="o">=</span> <span class="n">load_iris</span><span class="p">()</span>
<span class="n">X</span><span class="p">,</span> <span class="n">y</span> <span class="o">=</span> <span class="n">iris</span><span class="o">.</span><span class="n">data</span><span class="p">,</span> <span class="n">iris</span><span class="o">.</span><span class="n">target</span>
<span class="kn">from</span> <span class="nn">sklearn.model_selection</span> <span class="k">import</span> <span class="n">train_test_split</span>
<span class="n">X_train</span><span class="p">,</span> <span class="n">X_test</span><span class="p">,</span> <span class="n">y_train</span><span class="p">,</span> <span class="n">y_test</span> <span class="o">=</span> <span class="n">train_test_split</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">y</span><span class="p">)</span>
</pre></div>
</div>
<p>Then we fit a model.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">sklearn.linear_model</span> <span class="k">import</span> <span class="n">LogisticRegression</span>
<span class="n">clr</span> <span class="o">=</span> <span class="n">LogisticRegression</span><span class="p">()</span>
<span class="n">clr</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">X_train</span><span class="p">,</span> <span class="n">y_train</span><span class="p">)</span>
</pre></div>
</div>
<p class="sphx-glr-script-out">Out:</p>
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>LogisticRegression()
</pre></div>
</div>
<p>We compute the prediction on the test set
and we show the confusion matrix.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">sklearn.metrics</span> <span class="k">import</span> <span class="n">confusion_matrix</span>
<span class="n">pred</span> <span class="o">=</span> <span class="n">clr</span><span class="o">.</span><span class="n">predict</span><span class="p">(</span><span class="n">X_test</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">confusion_matrix</span><span class="p">(</span><span class="n">y_test</span><span class="p">,</span> <span class="n">pred</span><span class="p">))</span>
</pre></div>
</div>
<p class="sphx-glr-script-out">Out:</p>
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>[[13 0 0]
[ 0 10 1]
[ 0 0 14]]
</pre></div>
</div>
</section>
<section id="conversion-to-onnx-format">
<h2><a class="toc-backref" href="#id2">Conversion to ONNX format</a><a class="headerlink" href="#conversion-to-onnx-format" title="Permalink to this headline"></a></h2>
<p>We use module
<a class="reference external" href="https://github.com/onnx/sklearn-onnx">sklearn-onnx</a>
to convert the model into ONNX format.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">skl2onnx</span> <span class="k">import</span> <span class="n">convert_sklearn</span>
<span class="kn">from</span> <span class="nn">skl2onnx.common.data_types</span> <span class="k">import</span> <span class="n">FloatTensorType</span>
<span class="n">initial_type</span> <span class="o">=</span> <span class="p">[(</span><span class="s1">&#39;float_input&#39;</span><span class="p">,</span> <span class="n">FloatTensorType</span><span class="p">([</span><span class="kc">None</span><span class="p">,</span> <span class="mi">4</span><span class="p">]))]</span>
<span class="n">onx</span> <span class="o">=</span> <span class="n">convert_sklearn</span><span class="p">(</span><span class="n">clr</span><span class="p">,</span> <span class="n">initial_types</span><span class="o">=</span><span class="n">initial_type</span><span class="p">)</span>
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s2">&quot;logreg_iris.onnx&quot;</span><span class="p">,</span> <span class="s2">&quot;wb&quot;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
<span class="n">f</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">onx</span><span class="o">.</span><span class="n">SerializeToString</span><span class="p">())</span>
</pre></div>
</div>
<p>We load the model with ONNX Runtime and look at
its input and output.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">onnxruntime</span> <span class="k">as</span> <span class="nn">rt</span>
<span class="n">sess</span> <span class="o">=</span> <span class="n">rt</span><span class="o">.</span><span class="n">InferenceSession</span><span class="p">(</span><span class="s2">&quot;logreg_iris.onnx&quot;</span><span class="p">,</span> <span class="n">providers</span><span class="o">=</span><span class="n">rt</span><span class="o">.</span><span class="n">get_available_providers</span><span class="p">())</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;input name=&#39;</span><span class="si">{}</span><span class="s2">&#39; and shape=</span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span>
<span class="n">sess</span><span class="o">.</span><span class="n">get_inputs</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">sess</span><span class="o">.</span><span class="n">get_inputs</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">shape</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;output name=&#39;</span><span class="si">{}</span><span class="s2">&#39; and shape=</span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span>
<span class="n">sess</span><span class="o">.</span><span class="n">get_outputs</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">sess</span><span class="o">.</span><span class="n">get_outputs</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">shape</span><span class="p">))</span>
</pre></div>
</div>
<p class="sphx-glr-script-out">Out:</p>
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>input name=&#39;float_input&#39; and shape=[None, 4]
output name=&#39;output_label&#39; and shape=[None]
</pre></div>
</div>
<p>We compute the predictions.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">input_name</span> <span class="o">=</span> <span class="n">sess</span><span class="o">.</span><span class="n">get_inputs</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">name</span>
<span class="n">label_name</span> <span class="o">=</span> <span class="n">sess</span><span class="o">.</span><span class="n">get_outputs</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">name</span>
<span class="kn">import</span> <span class="nn">numpy</span>
<span class="n">pred_onx</span> <span class="o">=</span> <span class="n">sess</span><span class="o">.</span><span class="n">run</span><span class="p">([</span><span class="n">label_name</span><span class="p">],</span> <span class="p">{</span><span class="n">input_name</span><span class="p">:</span> <span class="n">X_test</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="n">numpy</span><span class="o">.</span><span class="n">float32</span><span class="p">)})[</span><span class="mi">0</span><span class="p">]</span>
<span class="nb">print</span><span class="p">(</span><span class="n">confusion_matrix</span><span class="p">(</span><span class="n">pred</span><span class="p">,</span> <span class="n">pred_onx</span><span class="p">))</span>
</pre></div>
</div>
<p class="sphx-glr-script-out">Out:</p>
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>[[13 0 0]
[ 0 10 0]
[ 0 0 15]]
</pre></div>
</div>
<p>The prediction are perfectly identical.</p>
</section>
<section id="probabilities">
<h2><a class="toc-backref" href="#id3">Probabilities</a><a class="headerlink" href="#probabilities" title="Permalink to this headline"></a></h2>
<p>Probabilities are needed to compute other
relevant metrics such as the ROC Curve.
Let’s see how to get them first with
scikit-learn.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">prob_sklearn</span> <span class="o">=</span> <span class="n">clr</span><span class="o">.</span><span class="n">predict_proba</span><span class="p">(</span><span class="n">X_test</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">prob_sklearn</span><span class="p">[:</span><span class="mi">3</span><span class="p">])</span>
</pre></div>
</div>
<p class="sphx-glr-script-out">Out:</p>
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>[[2.95951945e-05 5.69800366e-02 9.42990368e-01]
[1.84923705e-05 3.93636566e-02 9.60617851e-01]
[1.30004554e-02 8.03678159e-01 1.83321386e-01]]
</pre></div>
</div>
<p>And then with ONNX Runtime.
The probabilies appear to be</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">prob_name</span> <span class="o">=</span> <span class="n">sess</span><span class="o">.</span><span class="n">get_outputs</span><span class="p">()[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">name</span>
<span class="n">prob_rt</span> <span class="o">=</span> <span class="n">sess</span><span class="o">.</span><span class="n">run</span><span class="p">([</span><span class="n">prob_name</span><span class="p">],</span> <span class="p">{</span><span class="n">input_name</span><span class="p">:</span> <span class="n">X_test</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="n">numpy</span><span class="o">.</span><span class="n">float32</span><span class="p">)})[</span><span class="mi">0</span><span class="p">]</span>
<span class="kn">import</span> <span class="nn">pprint</span>
<span class="n">pprint</span><span class="o">.</span><span class="n">pprint</span><span class="p">(</span><span class="n">prob_rt</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="mi">3</span><span class="p">])</span>
</pre></div>
</div>
<p class="sphx-glr-script-out">Out:</p>
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>[{0: 2.9595235901069827e-05, 1: 0.05698008090257645, 2: 0.9429903030395508},
{0: 1.8492364688427188e-05, 1: 0.03936365991830826, 2: 0.9606178402900696},
{0: 0.013000461272895336, 1: 0.8036782741546631, 2: 0.1833212822675705}]
</pre></div>
</div>
<p>Let’s benchmark.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">timeit</span> <span class="k">import</span> <span class="n">Timer</span>
<span class="k">def</span> <span class="nf">speed</span><span class="p">(</span><span class="n">inst</span><span class="p">,</span> <span class="n">number</span><span class="o">=</span><span class="mi">10</span><span class="p">,</span> <span class="n">repeat</span><span class="o">=</span><span class="mi">20</span><span class="p">):</span>
<span class="n">timer</span> <span class="o">=</span> <span class="n">Timer</span><span class="p">(</span><span class="n">inst</span><span class="p">,</span> <span class="nb">globals</span><span class="o">=</span><span class="nb">globals</span><span class="p">())</span>
<span class="n">raw</span> <span class="o">=</span> <span class="n">numpy</span><span class="o">.</span><span class="n">array</span><span class="p">(</span><span class="n">timer</span><span class="o">.</span><span class="n">repeat</span><span class="p">(</span><span class="n">repeat</span><span class="p">,</span> <span class="n">number</span><span class="o">=</span><span class="n">number</span><span class="p">))</span>
<span class="n">ave</span> <span class="o">=</span> <span class="n">raw</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span> <span class="o">/</span> <span class="nb">len</span><span class="p">(</span><span class="n">raw</span><span class="p">)</span> <span class="o">/</span> <span class="n">number</span>
<span class="n">mi</span><span class="p">,</span> <span class="n">ma</span> <span class="o">=</span> <span class="n">raw</span><span class="o">.</span><span class="n">min</span><span class="p">()</span> <span class="o">/</span> <span class="n">number</span><span class="p">,</span> <span class="n">raw</span><span class="o">.</span><span class="n">max</span><span class="p">()</span> <span class="o">/</span> <span class="n">number</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Average </span><span class="si">%1.3g</span><span class="s2"> min=</span><span class="si">%1.3g</span><span class="s2"> max=</span><span class="si">%1.3g</span><span class="s2">&quot;</span> <span class="o">%</span> <span class="p">(</span><span class="n">ave</span><span class="p">,</span> <span class="n">mi</span><span class="p">,</span> <span class="n">ma</span><span class="p">))</span>
<span class="k">return</span> <span class="n">ave</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Execution time for clr.predict&quot;</span><span class="p">)</span>
<span class="n">speed</span><span class="p">(</span><span class="s2">&quot;clr.predict(X_test)&quot;</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Execution time for ONNX Runtime&quot;</span><span class="p">)</span>
<span class="n">speed</span><span class="p">(</span><span class="s2">&quot;sess.run([label_name], {input_name: X_test.astype(numpy.float32)})[0]&quot;</span><span class="p">)</span>
</pre></div>
</div>
<p class="sphx-glr-script-out">Out:</p>
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>Execution time for clr.predict
Average 4.38e-05 min=4.25e-05 max=6.07e-05
Execution time for ONNX Runtime
Average 1.97e-05 min=1.92e-05 max=2.46e-05
1.9671264999914226e-05
</pre></div>
</div>
<p>Let’s benchmark a scenario similar to what a webservice
experiences: the model has to do one prediction at a time
as opposed to a batch of prediction.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">loop</span><span class="p">(</span><span class="n">X_test</span><span class="p">,</span> <span class="n">fct</span><span class="p">,</span> <span class="n">n</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="n">nrow</span> <span class="o">=</span> <span class="n">X_test</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="k">if</span> <span class="n">n</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
<span class="n">n</span> <span class="o">=</span> <span class="n">nrow</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="n">n</span><span class="p">):</span>
<span class="n">im</span> <span class="o">=</span> <span class="n">i</span> <span class="o">%</span> <span class="n">nrow</span>
<span class="n">fct</span><span class="p">(</span><span class="n">X_test</span><span class="p">[</span><span class="n">im</span><span class="p">:</span> <span class="n">im</span><span class="o">+</span><span class="mi">1</span><span class="p">])</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Execution time for clr.predict&quot;</span><span class="p">)</span>
<span class="n">speed</span><span class="p">(</span><span class="s2">&quot;loop(X_test, clr.predict, 100)&quot;</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">sess_predict</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="k">return</span> <span class="n">sess</span><span class="o">.</span><span class="n">run</span><span class="p">([</span><span class="n">label_name</span><span class="p">],</span> <span class="p">{</span><span class="n">input_name</span><span class="p">:</span> <span class="n">x</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="n">numpy</span><span class="o">.</span><span class="n">float32</span><span class="p">)})[</span><span class="mi">0</span><span class="p">]</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Execution time for sess_predict&quot;</span><span class="p">)</span>
<span class="n">speed</span><span class="p">(</span><span class="s2">&quot;loop(X_test, sess_predict, 100)&quot;</span><span class="p">)</span>
</pre></div>
</div>
<p class="sphx-glr-script-out">Out:</p>
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>Execution time for clr.predict
Average 0.00404 min=0.00402 max=0.00409
Execution time for sess_predict
Average 0.000881 min=0.000874 max=0.000912
0.0008813192099997735
</pre></div>
</div>
<p>Let’s do the same for the probabilities.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Execution time for predict_proba&quot;</span><span class="p">)</span>
<span class="n">speed</span><span class="p">(</span><span class="s2">&quot;loop(X_test, clr.predict_proba, 100)&quot;</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">sess_predict_proba</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="k">return</span> <span class="n">sess</span><span class="o">.</span><span class="n">run</span><span class="p">([</span><span class="n">prob_name</span><span class="p">],</span> <span class="p">{</span><span class="n">input_name</span><span class="p">:</span> <span class="n">x</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="n">numpy</span><span class="o">.</span><span class="n">float32</span><span class="p">)})[</span><span class="mi">0</span><span class="p">]</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Execution time for sess_predict_proba&quot;</span><span class="p">)</span>
<span class="n">speed</span><span class="p">(</span><span class="s2">&quot;loop(X_test, sess_predict_proba, 100)&quot;</span><span class="p">)</span>
</pre></div>
</div>
<p class="sphx-glr-script-out">Out:</p>
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>Execution time for predict_proba
Average 0.00599 min=0.00597 max=0.00606
Execution time for sess_predict_proba
Average 0.000883 min=0.000876 max=0.000916
0.0008830492349999729
</pre></div>
</div>
<p>This second comparison is better as
ONNX Runtime, in this experience,
computes the label and the probabilities
in every case.</p>
</section>
<section id="benchmark-with-randomforest">
<h2><a class="toc-backref" href="#id4">Benchmark with RandomForest</a><a class="headerlink" href="#benchmark-with-randomforest" title="Permalink to this headline"></a></h2>
<p>We first train and save a model in ONNX format.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">sklearn.ensemble</span> <span class="k">import</span> <span class="n">RandomForestClassifier</span>
<span class="n">rf</span> <span class="o">=</span> <span class="n">RandomForestClassifier</span><span class="p">()</span>
<span class="n">rf</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">X_train</span><span class="p">,</span> <span class="n">y_train</span><span class="p">)</span>
<span class="n">initial_type</span> <span class="o">=</span> <span class="p">[(</span><span class="s1">&#39;float_input&#39;</span><span class="p">,</span> <span class="n">FloatTensorType</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">4</span><span class="p">]))]</span>
<span class="n">onx</span> <span class="o">=</span> <span class="n">convert_sklearn</span><span class="p">(</span><span class="n">rf</span><span class="p">,</span> <span class="n">initial_types</span><span class="o">=</span><span class="n">initial_type</span><span class="p">)</span>
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s2">&quot;rf_iris.onnx&quot;</span><span class="p">,</span> <span class="s2">&quot;wb&quot;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
<span class="n">f</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">onx</span><span class="o">.</span><span class="n">SerializeToString</span><span class="p">())</span>
</pre></div>
</div>
<p>We compare.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">sess</span> <span class="o">=</span> <span class="n">rt</span><span class="o">.</span><span class="n">InferenceSession</span><span class="p">(</span><span class="s2">&quot;rf_iris.onnx&quot;</span><span class="p">,</span> <span class="n">providers</span><span class="o">=</span><span class="n">rt</span><span class="o">.</span><span class="n">get_available_providers</span><span class="p">())</span>
<span class="k">def</span> <span class="nf">sess_predict_proba_rf</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="k">return</span> <span class="n">sess</span><span class="o">.</span><span class="n">run</span><span class="p">([</span><span class="n">prob_name</span><span class="p">],</span> <span class="p">{</span><span class="n">input_name</span><span class="p">:</span> <span class="n">x</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="n">numpy</span><span class="o">.</span><span class="n">float32</span><span class="p">)})[</span><span class="mi">0</span><span class="p">]</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Execution time for predict_proba&quot;</span><span class="p">)</span>
<span class="n">speed</span><span class="p">(</span><span class="s2">&quot;loop(X_test, rf.predict_proba, 100)&quot;</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Execution time for sess_predict_proba&quot;</span><span class="p">)</span>
<span class="n">speed</span><span class="p">(</span><span class="s2">&quot;loop(X_test, sess_predict_proba_rf, 100)&quot;</span><span class="p">)</span>
</pre></div>
</div>
<p class="sphx-glr-script-out">Out:</p>
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>Execution time for predict_proba
Average 0.717 min=0.715 max=0.72
Execution time for sess_predict_proba
Average 0.00108 min=0.00107 max=0.00111
0.0010817126199989956
</pre></div>
</div>
<p>Let’s see with different number of trees.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">measures</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">for</span> <span class="n">n_trees</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="mi">51</span><span class="p">,</span> <span class="mi">5</span><span class="p">):</span>
<span class="nb">print</span><span class="p">(</span><span class="n">n_trees</span><span class="p">)</span>
<span class="n">rf</span> <span class="o">=</span> <span class="n">RandomForestClassifier</span><span class="p">(</span><span class="n">n_estimators</span><span class="o">=</span><span class="n">n_trees</span><span class="p">)</span>
<span class="n">rf</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">X_train</span><span class="p">,</span> <span class="n">y_train</span><span class="p">)</span>
<span class="n">initial_type</span> <span class="o">=</span> <span class="p">[(</span><span class="s1">&#39;float_input&#39;</span><span class="p">,</span> <span class="n">FloatTensorType</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">4</span><span class="p">]))]</span>
<span class="n">onx</span> <span class="o">=</span> <span class="n">convert_sklearn</span><span class="p">(</span><span class="n">rf</span><span class="p">,</span> <span class="n">initial_types</span><span class="o">=</span><span class="n">initial_type</span><span class="p">)</span>
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s2">&quot;rf_iris_</span><span class="si">%d</span><span class="s2">.onnx&quot;</span> <span class="o">%</span> <span class="n">n_trees</span><span class="p">,</span> <span class="s2">&quot;wb&quot;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
<span class="n">f</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">onx</span><span class="o">.</span><span class="n">SerializeToString</span><span class="p">())</span>
<span class="n">sess</span> <span class="o">=</span> <span class="n">rt</span><span class="o">.</span><span class="n">InferenceSession</span><span class="p">(</span><span class="s2">&quot;rf_iris_</span><span class="si">%d</span><span class="s2">.onnx&quot;</span> <span class="o">%</span> <span class="n">n_trees</span><span class="p">,</span> <span class="n">providers</span><span class="o">=</span><span class="n">rt</span><span class="o">.</span><span class="n">get_available_providers</span><span class="p">())</span>
<span class="k">def</span> <span class="nf">sess_predict_proba_loop</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="k">return</span> <span class="n">sess</span><span class="o">.</span><span class="n">run</span><span class="p">([</span><span class="n">prob_name</span><span class="p">],</span> <span class="p">{</span><span class="n">input_name</span><span class="p">:</span> <span class="n">x</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="n">numpy</span><span class="o">.</span><span class="n">float32</span><span class="p">)})[</span><span class="mi">0</span><span class="p">]</span>
<span class="n">tsk</span> <span class="o">=</span> <span class="n">speed</span><span class="p">(</span><span class="s2">&quot;loop(X_test, rf.predict_proba, 100)&quot;</span><span class="p">,</span> <span class="n">number</span><span class="o">=</span><span class="mi">5</span><span class="p">,</span> <span class="n">repeat</span><span class="o">=</span><span class="mi">5</span><span class="p">)</span>
<span class="n">trt</span> <span class="o">=</span> <span class="n">speed</span><span class="p">(</span><span class="s2">&quot;loop(X_test, sess_predict_proba_loop, 100)&quot;</span><span class="p">,</span> <span class="n">number</span><span class="o">=</span><span class="mi">5</span><span class="p">,</span> <span class="n">repeat</span><span class="o">=</span><span class="mi">5</span><span class="p">)</span>
<span class="n">measures</span><span class="o">.</span><span class="n">append</span><span class="p">({</span><span class="s1">&#39;n_trees&#39;</span><span class="p">:</span> <span class="n">n_trees</span><span class="p">,</span> <span class="s1">&#39;sklearn&#39;</span><span class="p">:</span> <span class="n">tsk</span><span class="p">,</span> <span class="s1">&#39;rt&#39;</span><span class="p">:</span> <span class="n">trt</span><span class="p">})</span>
<span class="kn">from</span> <span class="nn">pandas</span> <span class="k">import</span> <span class="n">DataFrame</span>
<span class="n">df</span> <span class="o">=</span> <span class="n">DataFrame</span><span class="p">(</span><span class="n">measures</span><span class="p">)</span>
<span class="n">ax</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">x</span><span class="o">=</span><span class="s2">&quot;n_trees&quot;</span><span class="p">,</span> <span class="n">y</span><span class="o">=</span><span class="s2">&quot;sklearn&quot;</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s2">&quot;scikit-learn&quot;</span><span class="p">,</span> <span class="n">c</span><span class="o">=</span><span class="s2">&quot;blue&quot;</span><span class="p">,</span> <span class="n">logy</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">df</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">x</span><span class="o">=</span><span class="s2">&quot;n_trees&quot;</span><span class="p">,</span> <span class="n">y</span><span class="o">=</span><span class="s2">&quot;rt&quot;</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s2">&quot;onnxruntime&quot;</span><span class="p">,</span>
<span class="n">ax</span><span class="o">=</span><span class="n">ax</span><span class="p">,</span> <span class="n">c</span><span class="o">=</span><span class="s2">&quot;green&quot;</span><span class="p">,</span> <span class="n">logy</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">ax</span><span class="o">.</span><span class="n">set_xlabel</span><span class="p">(</span><span class="s2">&quot;Number of trees&quot;</span><span class="p">)</span>
<span class="n">ax</span><span class="o">.</span><span class="n">set_ylabel</span><span class="p">(</span><span class="s2">&quot;Prediction time (s)&quot;</span><span class="p">)</span>
<span class="n">ax</span><span class="o">.</span><span class="n">set_title</span><span class="p">(</span><span class="s2">&quot;Speed comparison between scikit-learn and ONNX Runtime</span><span class="se">\n</span><span class="s2">For a random forest on Iris dataset&quot;</span><span class="p">)</span>
<span class="n">ax</span><span class="o">.</span><span class="n">legend</span><span class="p">()</span>
</pre></div>
</div>
<img src="../images/sphx_glr_plot_train_convert_predict_001.png" srcset="../images/sphx_glr_plot_train_convert_predict_001.png" alt="Speed comparison between scikit-learn and ONNX Runtime For a random forest on Iris dataset" class = "sphx-glr-single-img"/><p class="sphx-glr-script-out">Out:</p>
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>5
Average 0.0637 min=0.0636 max=0.0639
Average 0.000869 min=0.000857 max=0.000899
10
Average 0.0982 min=0.098 max=0.0987
Average 0.000884 min=0.000873 max=0.00091
15
Average 0.133 min=0.133 max=0.133
Average 0.000895 min=0.000885 max=0.000921
20
Average 0.168 min=0.167 max=0.168
Average 0.000915 min=0.000907 max=0.00094
25
Average 0.202 min=0.201 max=0.203
Average 0.000921 min=0.000913 max=0.000948
30
Average 0.236 min=0.236 max=0.237
Average 0.000922 min=0.000914 max=0.000948
35
Average 0.271 min=0.271 max=0.271
Average 0.000941 min=0.000928 max=0.000967
40
Average 0.305 min=0.305 max=0.305
Average 0.000948 min=0.000934 max=0.000977
45
Average 0.34 min=0.339 max=0.34
Average 0.000983 min=0.000972 max=0.00101
50
Average 0.375 min=0.374 max=0.375
Average 0.000972 min=0.000966 max=0.000992
&lt;matplotlib.legend.Legend object at 0x7feaf2025c10&gt;
</pre></div>
</div>
<p class="sphx-glr-timing"><strong>Total running time of the script:</strong> ( 3 minutes 22.159 seconds)</p>
<div class="sphx-glr-footer class sphx-glr-footer-example docutils container" id="sphx-glr-download-auto-examples-plot-train-convert-predict-py">
<div class="sphx-glr-download sphx-glr-download-python docutils container">
<p><a class="reference download internal" download="" href="../downloads/c647c128e0cf2b3db04ce60b41ef1a14/plot_train_convert_predict.py"><code class="xref download docutils literal notranslate"><span class="pre">Download</span> <span class="pre">Python</span> <span class="pre">source</span> <span class="pre">code:</span> <span class="pre">plot_train_convert_predict.py</span></code></a></p>
</div>
<div class="sphx-glr-download sphx-glr-download-jupyter docutils container">
<p><a class="reference download internal" download="" href="../downloads/1680115d3d937dfbb2d86adb705d9c5d/plot_train_convert_predict.ipynb"><code class="xref download docutils literal notranslate"><span class="pre">Download</span> <span class="pre">Jupyter</span> <span class="pre">notebook:</span> <span class="pre">plot_train_convert_predict.ipynb</span></code></a></p>
</div>
</div>
<p class="sphx-glr-signature"><a class="reference external" href="https://sphinx-gallery.github.io">Gallery generated by Sphinx-Gallery</a></p>
</section>
</section>
</div>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<p class="logo"><a href="../index.html">
<img class="logo" src="../static/ONNX_Runtime_icon.png" alt="Logo"/>
</a></p>
<h1 class="logo"><a href="../index.html">ONNX Runtime</a></h1>
<h3>Navigation</h3>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../tutorial.html">Tutorial</a></li>
<li class="toctree-l1"><a class="reference internal" href="../api_summary.html">API Summary</a></li>
<li class="toctree-l1"><a class="reference internal" href="index.html">Gallery of examples</a></li>
</ul>
<div class="relations">
<h3>Related Topics</h3>
<ul>
<li><a href="../index.html">Documentation overview</a><ul>
<li><a href="index.html">Gallery of examples</a><ul>
<li>Previous: <a href="plot_common_errors.html" title="previous chapter">Common errors with onnxruntime</a></li>
</ul></li>
</ul></li>
</ul>
</div>
<div id="searchbox" style="display: none" role="search">
<h3 id="searchlabel">Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="../search.html" method="get">
<input type="text" name="q" aria-labelledby="searchlabel" autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false"/>
<input type="submit" value="Go" />
</form>
</div>
</div>
<script>$('#searchbox').show(0);</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="footer">
&copy;2018-2021, Microsoft.
|
Powered by <a href="http://sphinx-doc.org/">Sphinx 4.3.2</a>
&amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.12</a>
|
<a href="../sources/auto_examples/plot_train_convert_predict.rst.txt"
rel="nofollow">Page source</a>
</div>
</body>
</html>
Reference in a new issue View git blame Copy permalink