saymrwulf/onnxruntime
1
0
Fork 0
mirror of https://github.com/saymrwulf/onnxruntime.git synced 2026-07-14 18:12:05 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
onnxruntime/docs/api/python/auto_examples/plot_train_convert_predict.html
github-actions[bot] 0059cdcc5e
[Automated]: Update Python API docs (#11113)
2022-08-02 18:27:25 -07:00

440 lines
No EOL
39 KiB
HTML
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="generator" content="Docutils 0.17.1: http://docutils.sourceforge.net/" />
<title>Train, convert and predict with ONNX Runtime &#8212; ONNX Runtime 1.12.0 documentation</title>
<link rel="stylesheet" type="text/css" href="../static/pygments.css" />
<link rel="stylesheet" type="text/css" href="../static/alabaster.css" />
<link rel="stylesheet" type="text/css" href="../static/graphviz.css" />
<link rel="stylesheet" type="text/css" href="../static/sg_gallery.css" />
<link rel="stylesheet" type="text/css" href="../static/sg_gallery-binder.css" />
<link rel="stylesheet" type="text/css" href="../static/sg_gallery-dataframe.css" />
<link rel="stylesheet" type="text/css" href="../static/sg_gallery-rendered-html.css" />
<script data-url_root="../" id="documentation_options" src="../static/documentation_options.js"></script>
<script src="../static/jquery.js"></script>
<script src="../static/underscore.js"></script>
<script src="../static/doctools.js"></script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="prev" title="Common errors with onnxruntime" href="plot_common_errors.html" />
<link rel="stylesheet" href="../static/custom.css" type="text/css" />
<meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
</head><body>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<div class="sphx-glr-download-link-note admonition note">
<p class="admonition-title">Note</p>
<p>Click <a class="reference internal" href="#sphx-glr-download-auto-examples-plot-train-convert-predict-py"><span class="std std-ref">here</span></a>
to download the full example code</p>
</div>
<section class="sphx-glr-example-title" id="train-convert-and-predict-with-onnx-runtime">
<span id="l-logreg-example"></span><span id="sphx-glr-auto-examples-plot-train-convert-predict-py"></span><h1>Train, convert and predict with ONNX Runtime<a class="headerlink" href="#train-convert-and-predict-with-onnx-runtime" title="Permalink to this headline"></a></h1>
<p>This example demonstrates an end to end scenario
starting with the training of a machine learned model
to its use in its converted from.</p>
<div class="contents local topic" id="contents">
<ul class="simple">
<li><p><a class="reference internal" href="#train-a-logistic-regression" id="id1">Train a logistic regression</a></p></li>
<li><p><a class="reference internal" href="#conversion-to-onnx-format" id="id2">Conversion to ONNX format</a></p></li>
<li><p><a class="reference internal" href="#probabilities" id="id3">Probabilities</a></p></li>
<li><p><a class="reference internal" href="#benchmark-with-randomforest" id="id4">Benchmark with RandomForest</a></p></li>
</ul>
</div>
<section id="train-a-logistic-regression">
<h2><a class="toc-backref" href="#id1">Train a logistic regression</a><a class="headerlink" href="#train-a-logistic-regression" title="Permalink to this headline"></a></h2>
<p>The first step consists in retrieving the iris datset.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">sklearn.datasets</span> <span class="k">import</span> <span class="n">load_iris</span>
<span class="n">iris</span> <span class="o">=</span> <span class="n">load_iris</span><span class="p">()</span>
<span class="n">X</span><span class="p">,</span> <span class="n">y</span> <span class="o">=</span> <span class="n">iris</span><span class="o">.</span><span class="n">data</span><span class="p">,</span> <span class="n">iris</span><span class="o">.</span><span class="n">target</span>
<span class="kn">from</span> <span class="nn">sklearn.model_selection</span> <span class="k">import</span> <span class="n">train_test_split</span>
<span class="n">X_train</span><span class="p">,</span> <span class="n">X_test</span><span class="p">,</span> <span class="n">y_train</span><span class="p">,</span> <span class="n">y_test</span> <span class="o">=</span> <span class="n">train_test_split</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">y</span><span class="p">)</span>
</pre></div>
</div>
<p>Then we fit a model.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">sklearn.linear_model</span> <span class="k">import</span> <span class="n">LogisticRegression</span>
<span class="n">clr</span> <span class="o">=</span> <span class="n">LogisticRegression</span><span class="p">()</span>
<span class="n">clr</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">X_train</span><span class="p">,</span> <span class="n">y_train</span><span class="p">)</span>
</pre></div>
</div>
<p class="sphx-glr-script-out">Out:</p>
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>LogisticRegression()
</pre></div>
</div>
<p>We compute the prediction on the test set
and we show the confusion matrix.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">sklearn.metrics</span> <span class="k">import</span> <span class="n">confusion_matrix</span>
<span class="n">pred</span> <span class="o">=</span> <span class="n">clr</span><span class="o">.</span><span class="n">predict</span><span class="p">(</span><span class="n">X_test</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">confusion_matrix</span><span class="p">(</span><span class="n">y_test</span><span class="p">,</span> <span class="n">pred</span><span class="p">))</span>
</pre></div>
</div>
<p class="sphx-glr-script-out">Out:</p>
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>[[ 8 0 0]
[ 0 9 1]
[ 0 2 18]]
</pre></div>
</div>
</section>
<section id="conversion-to-onnx-format">
<h2><a class="toc-backref" href="#id2">Conversion to ONNX format</a><a class="headerlink" href="#conversion-to-onnx-format" title="Permalink to this headline"></a></h2>
<p>We use module
<a class="reference external" href="https://github.com/onnx/sklearn-onnx">sklearn-onnx</a>
to convert the model into ONNX format.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">skl2onnx</span> <span class="k">import</span> <span class="n">convert_sklearn</span>
<span class="kn">from</span> <span class="nn">skl2onnx.common.data_types</span> <span class="k">import</span> <span class="n">FloatTensorType</span>
<span class="n">initial_type</span> <span class="o">=</span> <span class="p">[(</span><span class="s1">&#39;float_input&#39;</span><span class="p">,</span> <span class="n">FloatTensorType</span><span class="p">([</span><span class="kc">None</span><span class="p">,</span> <span class="mi">4</span><span class="p">]))]</span>
<span class="n">onx</span> <span class="o">=</span> <span class="n">convert_sklearn</span><span class="p">(</span><span class="n">clr</span><span class="p">,</span> <span class="n">initial_types</span><span class="o">=</span><span class="n">initial_type</span><span class="p">)</span>
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s2">&quot;logreg_iris.onnx&quot;</span><span class="p">,</span> <span class="s2">&quot;wb&quot;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
<span class="n">f</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">onx</span><span class="o">.</span><span class="n">SerializeToString</span><span class="p">())</span>
</pre></div>
</div>
<p>We load the model with ONNX Runtime and look at
its input and output.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">onnxruntime</span> <span class="k">as</span> <span class="nn">rt</span>
<span class="n">sess</span> <span class="o">=</span> <span class="n">rt</span><span class="o">.</span><span class="n">InferenceSession</span><span class="p">(</span><span class="s2">&quot;logreg_iris.onnx&quot;</span><span class="p">,</span> <span class="n">providers</span><span class="o">=</span><span class="n">rt</span><span class="o">.</span><span class="n">get_available_providers</span><span class="p">())</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;input name=&#39;</span><span class="si">{}</span><span class="s2">&#39; and shape=</span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span>
<span class="n">sess</span><span class="o">.</span><span class="n">get_inputs</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">sess</span><span class="o">.</span><span class="n">get_inputs</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">shape</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;output name=&#39;</span><span class="si">{}</span><span class="s2">&#39; and shape=</span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span>
<span class="n">sess</span><span class="o">.</span><span class="n">get_outputs</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">sess</span><span class="o">.</span><span class="n">get_outputs</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">shape</span><span class="p">))</span>
</pre></div>
</div>
<p class="sphx-glr-script-out">Out:</p>
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>input name=&#39;float_input&#39; and shape=[None, 4]
output name=&#39;output_label&#39; and shape=[None]
</pre></div>
</div>
<p>We compute the predictions.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">input_name</span> <span class="o">=</span> <span class="n">sess</span><span class="o">.</span><span class="n">get_inputs</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">name</span>
<span class="n">label_name</span> <span class="o">=</span> <span class="n">sess</span><span class="o">.</span><span class="n">get_outputs</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">name</span>
<span class="kn">import</span> <span class="nn">numpy</span>
<span class="n">pred_onx</span> <span class="o">=</span> <span class="n">sess</span><span class="o">.</span><span class="n">run</span><span class="p">([</span><span class="n">label_name</span><span class="p">],</span> <span class="p">{</span><span class="n">input_name</span><span class="p">:</span> <span class="n">X_test</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="n">numpy</span><span class="o">.</span><span class="n">float32</span><span class="p">)})[</span><span class="mi">0</span><span class="p">]</span>
<span class="nb">print</span><span class="p">(</span><span class="n">confusion_matrix</span><span class="p">(</span><span class="n">pred</span><span class="p">,</span> <span class="n">pred_onx</span><span class="p">))</span>
</pre></div>
</div>
<p class="sphx-glr-script-out">Out:</p>
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>[[ 8 0 0]
[ 0 11 0]
[ 0 0 19]]
</pre></div>
</div>
<p>The prediction are perfectly identical.</p>
</section>
<section id="probabilities">
<h2><a class="toc-backref" href="#id3">Probabilities</a><a class="headerlink" href="#probabilities" title="Permalink to this headline"></a></h2>
<p>Probabilities are needed to compute other
relevant metrics such as the ROC Curve.
Let’s see how to get them first with
scikit-learn.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">prob_sklearn</span> <span class="o">=</span> <span class="n">clr</span><span class="o">.</span><span class="n">predict_proba</span><span class="p">(</span><span class="n">X_test</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">prob_sklearn</span><span class="p">[:</span><span class="mi">3</span><span class="p">])</span>
</pre></div>
</div>
<p class="sphx-glr-script-out">Out:</p>
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>[[2.50532892e-03 8.48834843e-01 1.48659828e-01]
[1.80424972e-04 2.23589233e-01 7.76230342e-01]
[9.47235933e-01 5.27628799e-02 1.18664935e-06]]
</pre></div>
</div>
<p>And then with ONNX Runtime.
The probabilies appear to be</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">prob_name</span> <span class="o">=</span> <span class="n">sess</span><span class="o">.</span><span class="n">get_outputs</span><span class="p">()[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">name</span>
<span class="n">prob_rt</span> <span class="o">=</span> <span class="n">sess</span><span class="o">.</span><span class="n">run</span><span class="p">([</span><span class="n">prob_name</span><span class="p">],</span> <span class="p">{</span><span class="n">input_name</span><span class="p">:</span> <span class="n">X_test</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="n">numpy</span><span class="o">.</span><span class="n">float32</span><span class="p">)})[</span><span class="mi">0</span><span class="p">]</span>
<span class="kn">import</span> <span class="nn">pprint</span>
<span class="n">pprint</span><span class="o">.</span><span class="n">pprint</span><span class="p">(</span><span class="n">prob_rt</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="mi">3</span><span class="p">])</span>
</pre></div>
</div>
<p class="sphx-glr-script-out">Out:</p>
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>[{0: 0.00250532990321517, 1: 0.8488348722457886, 2: 0.1486598700284958},
{0: 0.00018042475858237594, 1: 0.22358904778957367, 2: 0.77623051404953},
{0: 0.9472358822822571, 1: 0.052762895822525024, 2: 1.1866491149703506e-06}]
</pre></div>
</div>
<p>Let’s benchmark.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">timeit</span> <span class="k">import</span> <span class="n">Timer</span>
<span class="k">def</span> <span class="nf">speed</span><span class="p">(</span><span class="n">inst</span><span class="p">,</span> <span class="n">number</span><span class="o">=</span><span class="mi">10</span><span class="p">,</span> <span class="n">repeat</span><span class="o">=</span><span class="mi">20</span><span class="p">):</span>
<span class="n">timer</span> <span class="o">=</span> <span class="n">Timer</span><span class="p">(</span><span class="n">inst</span><span class="p">,</span> <span class="nb">globals</span><span class="o">=</span><span class="nb">globals</span><span class="p">())</span>
<span class="n">raw</span> <span class="o">=</span> <span class="n">numpy</span><span class="o">.</span><span class="n">array</span><span class="p">(</span><span class="n">timer</span><span class="o">.</span><span class="n">repeat</span><span class="p">(</span><span class="n">repeat</span><span class="p">,</span> <span class="n">number</span><span class="o">=</span><span class="n">number</span><span class="p">))</span>
<span class="n">ave</span> <span class="o">=</span> <span class="n">raw</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span> <span class="o">/</span> <span class="nb">len</span><span class="p">(</span><span class="n">raw</span><span class="p">)</span> <span class="o">/</span> <span class="n">number</span>
<span class="n">mi</span><span class="p">,</span> <span class="n">ma</span> <span class="o">=</span> <span class="n">raw</span><span class="o">.</span><span class="n">min</span><span class="p">()</span> <span class="o">/</span> <span class="n">number</span><span class="p">,</span> <span class="n">raw</span><span class="o">.</span><span class="n">max</span><span class="p">()</span> <span class="o">/</span> <span class="n">number</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Average </span><span class="si">%1.3g</span><span class="s2"> min=</span><span class="si">%1.3g</span><span class="s2"> max=</span><span class="si">%1.3g</span><span class="s2">&quot;</span> <span class="o">%</span> <span class="p">(</span><span class="n">ave</span><span class="p">,</span> <span class="n">mi</span><span class="p">,</span> <span class="n">ma</span><span class="p">))</span>
<span class="k">return</span> <span class="n">ave</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Execution time for clr.predict&quot;</span><span class="p">)</span>
<span class="n">speed</span><span class="p">(</span><span class="s2">&quot;clr.predict(X_test)&quot;</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Execution time for ONNX Runtime&quot;</span><span class="p">)</span>
<span class="n">speed</span><span class="p">(</span><span class="s2">&quot;sess.run([label_name], {input_name: X_test.astype(numpy.float32)})[0]&quot;</span><span class="p">)</span>
</pre></div>
</div>
<p class="sphx-glr-script-out">Out:</p>
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>Execution time for clr.predict
Average 5.1e-05 min=4.58e-05 max=6.63e-05
Execution time for ONNX Runtime
Average 2.42e-05 min=2.33e-05 max=3.53e-05
2.4194435000879365e-05
</pre></div>
</div>
<p>Let’s benchmark a scenario similar to what a webservice
experiences: the model has to do one prediction at a time
as opposed to a batch of prediction.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">loop</span><span class="p">(</span><span class="n">X_test</span><span class="p">,</span> <span class="n">fct</span><span class="p">,</span> <span class="n">n</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="n">nrow</span> <span class="o">=</span> <span class="n">X_test</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="k">if</span> <span class="n">n</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
<span class="n">n</span> <span class="o">=</span> <span class="n">nrow</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="n">n</span><span class="p">):</span>
<span class="n">im</span> <span class="o">=</span> <span class="n">i</span> <span class="o">%</span> <span class="n">nrow</span>
<span class="n">fct</span><span class="p">(</span><span class="n">X_test</span><span class="p">[</span><span class="n">im</span><span class="p">:</span> <span class="n">im</span><span class="o">+</span><span class="mi">1</span><span class="p">])</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Execution time for clr.predict&quot;</span><span class="p">)</span>
<span class="n">speed</span><span class="p">(</span><span class="s2">&quot;loop(X_test, clr.predict, 100)&quot;</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">sess_predict</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="k">return</span> <span class="n">sess</span><span class="o">.</span><span class="n">run</span><span class="p">([</span><span class="n">label_name</span><span class="p">],</span> <span class="p">{</span><span class="n">input_name</span><span class="p">:</span> <span class="n">x</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="n">numpy</span><span class="o">.</span><span class="n">float32</span><span class="p">)})[</span><span class="mi">0</span><span class="p">]</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Execution time for sess_predict&quot;</span><span class="p">)</span>
<span class="n">speed</span><span class="p">(</span><span class="s2">&quot;loop(X_test, sess_predict, 100)&quot;</span><span class="p">)</span>
</pre></div>
</div>
<p class="sphx-glr-script-out">Out:</p>
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>Execution time for clr.predict
Average 0.00499 min=0.00471 max=0.0057
Execution time for sess_predict
Average 0.00116 min=0.00105 max=0.00138
0.001157615160000205
</pre></div>
</div>
<p>Let’s do the same for the probabilities.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Execution time for predict_proba&quot;</span><span class="p">)</span>
<span class="n">speed</span><span class="p">(</span><span class="s2">&quot;loop(X_test, clr.predict_proba, 100)&quot;</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">sess_predict_proba</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="k">return</span> <span class="n">sess</span><span class="o">.</span><span class="n">run</span><span class="p">([</span><span class="n">prob_name</span><span class="p">],</span> <span class="p">{</span><span class="n">input_name</span><span class="p">:</span> <span class="n">x</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="n">numpy</span><span class="o">.</span><span class="n">float32</span><span class="p">)})[</span><span class="mi">0</span><span class="p">]</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Execution time for sess_predict_proba&quot;</span><span class="p">)</span>
<span class="n">speed</span><span class="p">(</span><span class="s2">&quot;loop(X_test, sess_predict_proba, 100)&quot;</span><span class="p">)</span>
</pre></div>
</div>
<p class="sphx-glr-script-out">Out:</p>
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>Execution time for predict_proba
Average 0.00729 min=0.00697 max=0.00805
Execution time for sess_predict_proba
Average 0.00109 min=0.00106 max=0.00114
0.0010851338200001236
</pre></div>
</div>
<p>This second comparison is better as
ONNX Runtime, in this experience,
computes the label and the probabilities
in every case.</p>
</section>
<section id="benchmark-with-randomforest">
<h2><a class="toc-backref" href="#id4">Benchmark with RandomForest</a><a class="headerlink" href="#benchmark-with-randomforest" title="Permalink to this headline"></a></h2>
<p>We first train and save a model in ONNX format.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">sklearn.ensemble</span> <span class="k">import</span> <span class="n">RandomForestClassifier</span>
<span class="n">rf</span> <span class="o">=</span> <span class="n">RandomForestClassifier</span><span class="p">()</span>
<span class="n">rf</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">X_train</span><span class="p">,</span> <span class="n">y_train</span><span class="p">)</span>
<span class="n">initial_type</span> <span class="o">=</span> <span class="p">[(</span><span class="s1">&#39;float_input&#39;</span><span class="p">,</span> <span class="n">FloatTensorType</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">4</span><span class="p">]))]</span>
<span class="n">onx</span> <span class="o">=</span> <span class="n">convert_sklearn</span><span class="p">(</span><span class="n">rf</span><span class="p">,</span> <span class="n">initial_types</span><span class="o">=</span><span class="n">initial_type</span><span class="p">)</span>
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s2">&quot;rf_iris.onnx&quot;</span><span class="p">,</span> <span class="s2">&quot;wb&quot;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
<span class="n">f</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">onx</span><span class="o">.</span><span class="n">SerializeToString</span><span class="p">())</span>
</pre></div>
</div>
<p>We compare.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">sess</span> <span class="o">=</span> <span class="n">rt</span><span class="o">.</span><span class="n">InferenceSession</span><span class="p">(</span><span class="s2">&quot;rf_iris.onnx&quot;</span><span class="p">,</span> <span class="n">providers</span><span class="o">=</span><span class="n">rt</span><span class="o">.</span><span class="n">get_available_providers</span><span class="p">())</span>
<span class="k">def</span> <span class="nf">sess_predict_proba_rf</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="k">return</span> <span class="n">sess</span><span class="o">.</span><span class="n">run</span><span class="p">([</span><span class="n">prob_name</span><span class="p">],</span> <span class="p">{</span><span class="n">input_name</span><span class="p">:</span> <span class="n">x</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="n">numpy</span><span class="o">.</span><span class="n">float32</span><span class="p">)})[</span><span class="mi">0</span><span class="p">]</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Execution time for predict_proba&quot;</span><span class="p">)</span>
<span class="n">speed</span><span class="p">(</span><span class="s2">&quot;loop(X_test, rf.predict_proba, 100)&quot;</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Execution time for sess_predict_proba&quot;</span><span class="p">)</span>
<span class="n">speed</span><span class="p">(</span><span class="s2">&quot;loop(X_test, sess_predict_proba_rf, 100)&quot;</span><span class="p">)</span>
</pre></div>
</div>
<p class="sphx-glr-script-out">Out:</p>
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>Execution time for predict_proba
Average 0.845 min=0.839 max=0.853
Execution time for sess_predict_proba
Average 0.00131 min=0.00129 max=0.00133
0.0013066128300005175
</pre></div>
</div>
<p>Let’s see with different number of trees.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">measures</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">for</span> <span class="n">n_trees</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="mi">51</span><span class="p">,</span> <span class="mi">5</span><span class="p">):</span>
<span class="nb">print</span><span class="p">(</span><span class="n">n_trees</span><span class="p">)</span>
<span class="n">rf</span> <span class="o">=</span> <span class="n">RandomForestClassifier</span><span class="p">(</span><span class="n">n_estimators</span><span class="o">=</span><span class="n">n_trees</span><span class="p">)</span>
<span class="n">rf</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">X_train</span><span class="p">,</span> <span class="n">y_train</span><span class="p">)</span>
<span class="n">initial_type</span> <span class="o">=</span> <span class="p">[(</span><span class="s1">&#39;float_input&#39;</span><span class="p">,</span> <span class="n">FloatTensorType</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">4</span><span class="p">]))]</span>
<span class="n">onx</span> <span class="o">=</span> <span class="n">convert_sklearn</span><span class="p">(</span><span class="n">rf</span><span class="p">,</span> <span class="n">initial_types</span><span class="o">=</span><span class="n">initial_type</span><span class="p">)</span>
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s2">&quot;rf_iris_</span><span class="si">%d</span><span class="s2">.onnx&quot;</span> <span class="o">%</span> <span class="n">n_trees</span><span class="p">,</span> <span class="s2">&quot;wb&quot;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
<span class="n">f</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">onx</span><span class="o">.</span><span class="n">SerializeToString</span><span class="p">())</span>
<span class="n">sess</span> <span class="o">=</span> <span class="n">rt</span><span class="o">.</span><span class="n">InferenceSession</span><span class="p">(</span><span class="s2">&quot;rf_iris_</span><span class="si">%d</span><span class="s2">.onnx&quot;</span> <span class="o">%</span> <span class="n">n_trees</span><span class="p">,</span> <span class="n">providers</span><span class="o">=</span><span class="n">rt</span><span class="o">.</span><span class="n">get_available_providers</span><span class="p">())</span>
<span class="k">def</span> <span class="nf">sess_predict_proba_loop</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="k">return</span> <span class="n">sess</span><span class="o">.</span><span class="n">run</span><span class="p">([</span><span class="n">prob_name</span><span class="p">],</span> <span class="p">{</span><span class="n">input_name</span><span class="p">:</span> <span class="n">x</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="n">numpy</span><span class="o">.</span><span class="n">float32</span><span class="p">)})[</span><span class="mi">0</span><span class="p">]</span>
<span class="n">tsk</span> <span class="o">=</span> <span class="n">speed</span><span class="p">(</span><span class="s2">&quot;loop(X_test, rf.predict_proba, 100)&quot;</span><span class="p">,</span> <span class="n">number</span><span class="o">=</span><span class="mi">5</span><span class="p">,</span> <span class="n">repeat</span><span class="o">=</span><span class="mi">5</span><span class="p">)</span>
<span class="n">trt</span> <span class="o">=</span> <span class="n">speed</span><span class="p">(</span><span class="s2">&quot;loop(X_test, sess_predict_proba_loop, 100)&quot;</span><span class="p">,</span> <span class="n">number</span><span class="o">=</span><span class="mi">5</span><span class="p">,</span> <span class="n">repeat</span><span class="o">=</span><span class="mi">5</span><span class="p">)</span>
<span class="n">measures</span><span class="o">.</span><span class="n">append</span><span class="p">({</span><span class="s1">&#39;n_trees&#39;</span><span class="p">:</span> <span class="n">n_trees</span><span class="p">,</span> <span class="s1">&#39;sklearn&#39;</span><span class="p">:</span> <span class="n">tsk</span><span class="p">,</span> <span class="s1">&#39;rt&#39;</span><span class="p">:</span> <span class="n">trt</span><span class="p">})</span>
<span class="kn">from</span> <span class="nn">pandas</span> <span class="k">import</span> <span class="n">DataFrame</span>
<span class="n">df</span> <span class="o">=</span> <span class="n">DataFrame</span><span class="p">(</span><span class="n">measures</span><span class="p">)</span>
<span class="n">ax</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">x</span><span class="o">=</span><span class="s2">&quot;n_trees&quot;</span><span class="p">,</span> <span class="n">y</span><span class="o">=</span><span class="s2">&quot;sklearn&quot;</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s2">&quot;scikit-learn&quot;</span><span class="p">,</span> <span class="n">c</span><span class="o">=</span><span class="s2">&quot;blue&quot;</span><span class="p">,</span> <span class="n">logy</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">df</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">x</span><span class="o">=</span><span class="s2">&quot;n_trees&quot;</span><span class="p">,</span> <span class="n">y</span><span class="o">=</span><span class="s2">&quot;rt&quot;</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s2">&quot;onnxruntime&quot;</span><span class="p">,</span>
<span class="n">ax</span><span class="o">=</span><span class="n">ax</span><span class="p">,</span> <span class="n">c</span><span class="o">=</span><span class="s2">&quot;green&quot;</span><span class="p">,</span> <span class="n">logy</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">ax</span><span class="o">.</span><span class="n">set_xlabel</span><span class="p">(</span><span class="s2">&quot;Number of trees&quot;</span><span class="p">)</span>
<span class="n">ax</span><span class="o">.</span><span class="n">set_ylabel</span><span class="p">(</span><span class="s2">&quot;Prediction time (s)&quot;</span><span class="p">)</span>
<span class="n">ax</span><span class="o">.</span><span class="n">set_title</span><span class="p">(</span><span class="s2">&quot;Speed comparison between scikit-learn and ONNX Runtime</span><span class="se">\n</span><span class="s2">For a random forest on Iris dataset&quot;</span><span class="p">)</span>
<span class="n">ax</span><span class="o">.</span><span class="n">legend</span><span class="p">()</span>
</pre></div>
</div>
<img src="../images/sphx_glr_plot_train_convert_predict_001.png" srcset="../images/sphx_glr_plot_train_convert_predict_001.png" alt="Speed comparison between scikit-learn and ONNX Runtime For a random forest on Iris dataset" class = "sphx-glr-single-img"/><p class="sphx-glr-script-out">Out:</p>
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>5
Average 0.0745 min=0.0741 max=0.0748
Average 0.00103 min=0.000996 max=0.00105
10
Average 0.115 min=0.114 max=0.116
Average 0.00103 min=0.001 max=0.00106
15
Average 0.156 min=0.154 max=0.157
Average 0.00109 min=0.00107 max=0.00112
20
Average 0.197 min=0.196 max=0.197
Average 0.00108 min=0.00105 max=0.00113
25
Average 0.236 min=0.235 max=0.238
Average 0.00109 min=0.00106 max=0.00113
30
Average 0.277 min=0.277 max=0.278
Average 0.00112 min=0.00107 max=0.00116
35
Average 0.319 min=0.318 max=0.321
Average 0.00114 min=0.00113 max=0.00116
40
Average 0.359 min=0.358 max=0.361
Average 0.00114 min=0.0011 max=0.00118
45
Average 0.399 min=0.397 max=0.401
Average 0.00116 min=0.00113 max=0.00118
50
Average 0.439 min=0.437 max=0.441
Average 0.00116 min=0.00115 max=0.00121
&lt;matplotlib.legend.Legend object at 0x7fa5848e9040&gt;
</pre></div>
</div>
<p class="sphx-glr-timing"><strong>Total running time of the script:</strong> ( 3 minutes 57.930 seconds)</p>
<div class="sphx-glr-footer class sphx-glr-footer-example docutils container" id="sphx-glr-download-auto-examples-plot-train-convert-predict-py">
<div class="sphx-glr-download sphx-glr-download-python docutils container">
<p><a class="reference download internal" download="" href="../downloads/c647c128e0cf2b3db04ce60b41ef1a14/plot_train_convert_predict.py"><code class="xref download docutils literal notranslate"><span class="pre">Download</span> <span class="pre">Python</span> <span class="pre">source</span> <span class="pre">code:</span> <span class="pre">plot_train_convert_predict.py</span></code></a></p>
</div>
<div class="sphx-glr-download sphx-glr-download-jupyter docutils container">
<p><a class="reference download internal" download="" href="../downloads/1680115d3d937dfbb2d86adb705d9c5d/plot_train_convert_predict.ipynb"><code class="xref download docutils literal notranslate"><span class="pre">Download</span> <span class="pre">Jupyter</span> <span class="pre">notebook:</span> <span class="pre">plot_train_convert_predict.ipynb</span></code></a></p>
</div>
</div>
<p class="sphx-glr-signature"><a class="reference external" href="https://sphinx-gallery.github.io">Gallery generated by Sphinx-Gallery</a></p>
</section>
</section>
</div>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<p class="logo"><a href="../index.html">
<img class="logo" src="../static/ONNX_Runtime_icon.png" alt="Logo"/>
</a></p>
<h1 class="logo"><a href="../index.html">ONNX Runtime</a></h1>
<h3>Navigation</h3>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../tutorial.html">Tutorial</a></li>
<li class="toctree-l1"><a class="reference internal" href="../api_summary.html">API</a></li>
<li class="toctree-l1"><a class="reference internal" href="index.html">Gallery of examples</a></li>
</ul>
<div class="relations">
<h3>Related Topics</h3>
<ul>
<li><a href="../index.html">Documentation overview</a><ul>
<li><a href="index.html">Gallery of examples</a><ul>
<li>Previous: <a href="plot_common_errors.html" title="previous chapter">Common errors with onnxruntime</a></li>
</ul></li>
</ul></li>
</ul>
</div>
<div id="searchbox" style="display: none" role="search">
<h3 id="searchlabel">Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="../search.html" method="get">
<input type="text" name="q" aria-labelledby="searchlabel" autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false"/>
<input type="submit" value="Go" />
</form>
</div>
</div>
<script>$('#searchbox').show(0);</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="footer">
&copy;2018-2021, Microsoft.
|
Powered by <a href="http://sphinx-doc.org/">Sphinx 4.5.0</a>
&amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.12</a>
|
<a href="../sources/auto_examples/plot_train_convert_predict.rst.txt"
rel="nofollow">Page source</a>
</div>
</body>
</html>
Reference in a new issue View git blame Copy permalink