mirror of
https://github.com/saymrwulf/onnxruntime.git
synced 2026-07-14 18:12:05 +00:00
a0f3e30de6
* initial setup and rename "how to" to "setup" * move API to main nav * move api to main nav * add get starated, rework nav order * rename to install move mds out of install section * update api nav and home page * add install docs and python qs updates * python get started work * remove c and obj c for now * move java, python, and obj-c docs under api folder * move java api html to iframe (ugh) * remove api docs w/o details, move api text getstar * remove api docs wo detail updates get started * remvoe iframes * move eco system to main nav * fix api buttons * added more examples moved intro to ORT * fix links * fix get started titles * fix get started titles * fix more links * fix more links * more link fixes * fix nav remove inferencing and training subnav * fix top nav remove inference and training nav * fix title * fix tutorials nav hierarchy * fix python api button * add tenorflow keras example * fix quickstart toc * add imports fix spacing * fix links * update nav and python get started page * move ort training example, add coming soon for iot * update C# get started * fix spacing on quantization * Add some js get started content * fix formatting * fix typo * removed onnx-pytorch and onnx-tf * updated pip install torch and added links iot page * added pytorch tutorial heirarchy * updated web to docs soon added release blog link * add web link
449 lines
No EOL
39 KiB
HTML
449 lines
No EOL
39 KiB
HTML
|
||
<!DOCTYPE html>
|
||
|
||
<html lang="en">
|
||
<head>
|
||
<meta charset="utf-8" />
|
||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||
<title>Train, convert and predict with ONNX Runtime — ONNX Runtime 1.7.0 documentation</title>
|
||
<link rel="stylesheet" href="../static/pygments.css" type="text/css" />
|
||
<link rel="stylesheet" href="../static/alabaster.css" type="text/css" />
|
||
<link rel="stylesheet" type="text/css" href="../static/graphviz.css" />
|
||
<link rel="stylesheet" type="text/css" href="../static/gallery.css" />
|
||
<link rel="stylesheet" type="text/css" href="../static/gallery-binder.css" />
|
||
<link rel="stylesheet" type="text/css" href="../static/gallery-dataframe.css" />
|
||
<link rel="stylesheet" type="text/css" href="../static/gallery-rendered-html.css" />
|
||
<script id="documentation_options" data-url_root="../" src="../static/documentation_options.js"></script>
|
||
<script src="../static/jquery.js"></script>
|
||
<script src="../static/underscore.js"></script>
|
||
<script src="../static/doctools.js"></script>
|
||
<link rel="index" title="Index" href="../genindex.html" />
|
||
<link rel="search" title="Search" href="../search.html" />
|
||
<link rel="prev" title="Common errors with onnxruntime" href="plot_common_errors.html" />
|
||
|
||
<link rel="stylesheet" href="../static/custom.css" type="text/css" />
|
||
|
||
|
||
<meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
|
||
|
||
</head><body>
|
||
|
||
|
||
<div class="document">
|
||
<div class="documentwrapper">
|
||
<div class="bodywrapper">
|
||
|
||
|
||
<div class="body" role="main">
|
||
|
||
<div class="sphx-glr-download-link-note admonition note">
|
||
<p class="admonition-title">Note</p>
|
||
<p>Click <a class="reference internal" href="#sphx-glr-download-auto-examples-plot-train-convert-predict-py"><span class="std std-ref">here</span></a>
|
||
to download the full example code</p>
|
||
</div>
|
||
<div class="sphx-glr-example-title section" id="train-convert-and-predict-with-onnx-runtime">
|
||
<span id="l-logreg-example"></span><span id="sphx-glr-auto-examples-plot-train-convert-predict-py"></span><h1>Train, convert and predict with ONNX Runtime<a class="headerlink" href="#train-convert-and-predict-with-onnx-runtime" title="Permalink to this headline">¶</a></h1>
|
||
<p>This example demonstrates an end to end scenario
|
||
starting with the training of a machine learned model
|
||
to its use in its converted from.</p>
|
||
<div class="contents local topic" id="contents">
|
||
<ul class="simple">
|
||
<li><p><a class="reference internal" href="#train-a-logistic-regression" id="id1">Train a logistic regression</a></p></li>
|
||
<li><p><a class="reference internal" href="#conversion-to-onnx-format" id="id2">Conversion to ONNX format</a></p></li>
|
||
<li><p><a class="reference internal" href="#probabilities" id="id3">Probabilities</a></p></li>
|
||
<li><p><a class="reference internal" href="#benchmark-with-randomforest" id="id4">Benchmark with RandomForest</a></p></li>
|
||
</ul>
|
||
</div>
|
||
<div class="section" id="train-a-logistic-regression">
|
||
<h2><a class="toc-backref" href="#id1">Train a logistic regression</a><a class="headerlink" href="#train-a-logistic-regression" title="Permalink to this headline">¶</a></h2>
|
||
<p>The first step consists in retrieving the iris datset.</p>
|
||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">sklearn.datasets</span> <span class="kn">import</span> <span class="n">load_iris</span>
|
||
<span class="n">iris</span> <span class="o">=</span> <span class="n">load_iris</span><span class="p">()</span>
|
||
<span class="n">X</span><span class="p">,</span> <span class="n">y</span> <span class="o">=</span> <span class="n">iris</span><span class="o">.</span><span class="n">data</span><span class="p">,</span> <span class="n">iris</span><span class="o">.</span><span class="n">target</span>
|
||
|
||
<span class="kn">from</span> <span class="nn">sklearn.model_selection</span> <span class="kn">import</span> <span class="n">train_test_split</span>
|
||
<span class="n">X_train</span><span class="p">,</span> <span class="n">X_test</span><span class="p">,</span> <span class="n">y_train</span><span class="p">,</span> <span class="n">y_test</span> <span class="o">=</span> <span class="n">train_test_split</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">y</span><span class="p">)</span>
|
||
</pre></div>
|
||
</div>
|
||
<p>Then we fit a model.</p>
|
||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">sklearn.linear_model</span> <span class="kn">import</span> <span class="n">LogisticRegression</span>
|
||
<span class="n">clr</span> <span class="o">=</span> <span class="n">LogisticRegression</span><span class="p">()</span>
|
||
<span class="n">clr</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">X_train</span><span class="p">,</span> <span class="n">y_train</span><span class="p">)</span>
|
||
</pre></div>
|
||
</div>
|
||
<p class="sphx-glr-script-out">Out:</p>
|
||
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>/opt/miniconda/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:765: ConvergenceWarning: lbfgs failed to converge (status=1):
|
||
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
|
||
|
||
Increase the number of iterations (max_iter) or scale the data as shown in:
|
||
https://scikit-learn.org/stable/modules/preprocessing.html
|
||
Please also refer to the documentation for alternative solver options:
|
||
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
|
||
extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)
|
||
|
||
LogisticRegression()
|
||
</pre></div>
|
||
</div>
|
||
<p>We compute the prediction on the test set
|
||
and we show the confusion matrix.</p>
|
||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">sklearn.metrics</span> <span class="kn">import</span> <span class="n">confusion_matrix</span>
|
||
|
||
<span class="n">pred</span> <span class="o">=</span> <span class="n">clr</span><span class="o">.</span><span class="n">predict</span><span class="p">(</span><span class="n">X_test</span><span class="p">)</span>
|
||
<span class="nb">print</span><span class="p">(</span><span class="n">confusion_matrix</span><span class="p">(</span><span class="n">y_test</span><span class="p">,</span> <span class="n">pred</span><span class="p">))</span>
|
||
</pre></div>
|
||
</div>
|
||
<p class="sphx-glr-script-out">Out:</p>
|
||
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>[[10 0 0]
|
||
[ 0 14 0]
|
||
[ 0 1 13]]
|
||
</pre></div>
|
||
</div>
|
||
</div>
|
||
<div class="section" id="conversion-to-onnx-format">
|
||
<h2><a class="toc-backref" href="#id2">Conversion to ONNX format</a><a class="headerlink" href="#conversion-to-onnx-format" title="Permalink to this headline">¶</a></h2>
|
||
<p>We use module
|
||
<a class="reference external" href="https://github.com/onnx/sklearn-onnx">sklearn-onnx</a>
|
||
to convert the model into ONNX format.</p>
|
||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">skl2onnx</span> <span class="kn">import</span> <span class="n">convert_sklearn</span>
|
||
<span class="kn">from</span> <span class="nn">skl2onnx.common.data_types</span> <span class="kn">import</span> <span class="n">FloatTensorType</span>
|
||
|
||
<span class="n">initial_type</span> <span class="o">=</span> <span class="p">[(</span><span class="s1">'float_input'</span><span class="p">,</span> <span class="n">FloatTensorType</span><span class="p">([</span><span class="kc">None</span><span class="p">,</span> <span class="mi">4</span><span class="p">]))]</span>
|
||
<span class="n">onx</span> <span class="o">=</span> <span class="n">convert_sklearn</span><span class="p">(</span><span class="n">clr</span><span class="p">,</span> <span class="n">initial_types</span><span class="o">=</span><span class="n">initial_type</span><span class="p">)</span>
|
||
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s2">"logreg_iris.onnx"</span><span class="p">,</span> <span class="s2">"wb"</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
|
||
<span class="n">f</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">onx</span><span class="o">.</span><span class="n">SerializeToString</span><span class="p">())</span>
|
||
</pre></div>
|
||
</div>
|
||
<p>We load the model with ONNX Runtime and look at
|
||
its input and output.</p>
|
||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">onnxruntime</span> <span class="k">as</span> <span class="nn">rt</span>
|
||
<span class="n">sess</span> <span class="o">=</span> <span class="n">rt</span><span class="o">.</span><span class="n">InferenceSession</span><span class="p">(</span><span class="s2">"logreg_iris.onnx"</span><span class="p">)</span>
|
||
|
||
<span class="nb">print</span><span class="p">(</span><span class="s2">"input name='</span><span class="si">{}</span><span class="s2">' and shape=</span><span class="si">{}</span><span class="s2">"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span>
|
||
<span class="n">sess</span><span class="o">.</span><span class="n">get_inputs</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">sess</span><span class="o">.</span><span class="n">get_inputs</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">shape</span><span class="p">))</span>
|
||
<span class="nb">print</span><span class="p">(</span><span class="s2">"output name='</span><span class="si">{}</span><span class="s2">' and shape=</span><span class="si">{}</span><span class="s2">"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span>
|
||
<span class="n">sess</span><span class="o">.</span><span class="n">get_outputs</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="n">sess</span><span class="o">.</span><span class="n">get_outputs</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">shape</span><span class="p">))</span>
|
||
</pre></div>
|
||
</div>
|
||
<p class="sphx-glr-script-out">Out:</p>
|
||
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>input name='float_input' and shape=[None, 4]
|
||
output name='output_label' and shape=[None]
|
||
</pre></div>
|
||
</div>
|
||
<p>We compute the predictions.</p>
|
||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">input_name</span> <span class="o">=</span> <span class="n">sess</span><span class="o">.</span><span class="n">get_inputs</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">name</span>
|
||
<span class="n">label_name</span> <span class="o">=</span> <span class="n">sess</span><span class="o">.</span><span class="n">get_outputs</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">name</span>
|
||
|
||
<span class="kn">import</span> <span class="nn">numpy</span>
|
||
<span class="n">pred_onx</span> <span class="o">=</span> <span class="n">sess</span><span class="o">.</span><span class="n">run</span><span class="p">([</span><span class="n">label_name</span><span class="p">],</span> <span class="p">{</span><span class="n">input_name</span><span class="p">:</span> <span class="n">X_test</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="n">numpy</span><span class="o">.</span><span class="n">float32</span><span class="p">)})[</span><span class="mi">0</span><span class="p">]</span>
|
||
<span class="nb">print</span><span class="p">(</span><span class="n">confusion_matrix</span><span class="p">(</span><span class="n">pred</span><span class="p">,</span> <span class="n">pred_onx</span><span class="p">))</span>
|
||
</pre></div>
|
||
</div>
|
||
<p class="sphx-glr-script-out">Out:</p>
|
||
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>[[10 0 0]
|
||
[ 0 15 0]
|
||
[ 0 0 13]]
|
||
</pre></div>
|
||
</div>
|
||
<p>The prediction are perfectly identical.</p>
|
||
</div>
|
||
<div class="section" id="probabilities">
|
||
<h2><a class="toc-backref" href="#id3">Probabilities</a><a class="headerlink" href="#probabilities" title="Permalink to this headline">¶</a></h2>
|
||
<p>Probabilities are needed to compute other
|
||
relevant metrics such as the ROC Curve.
|
||
Let’s see how to get them first with
|
||
scikit-learn.</p>
|
||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">prob_sklearn</span> <span class="o">=</span> <span class="n">clr</span><span class="o">.</span><span class="n">predict_proba</span><span class="p">(</span><span class="n">X_test</span><span class="p">)</span>
|
||
<span class="nb">print</span><span class="p">(</span><span class="n">prob_sklearn</span><span class="p">[:</span><span class="mi">3</span><span class="p">])</span>
|
||
</pre></div>
|
||
</div>
|
||
<p class="sphx-glr-script-out">Out:</p>
|
||
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>[[4.15987711e-03 8.54898335e-01 1.40941788e-01]
|
||
[9.44228260e-01 5.57707615e-02 9.78002823e-07]
|
||
[5.17324282e-02 8.88396143e-01 5.98714288e-02]]
|
||
</pre></div>
|
||
</div>
|
||
<p>And then with ONNX Runtime.
|
||
The probabilies appear to be</p>
|
||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">prob_name</span> <span class="o">=</span> <span class="n">sess</span><span class="o">.</span><span class="n">get_outputs</span><span class="p">()[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">name</span>
|
||
<span class="n">prob_rt</span> <span class="o">=</span> <span class="n">sess</span><span class="o">.</span><span class="n">run</span><span class="p">([</span><span class="n">prob_name</span><span class="p">],</span> <span class="p">{</span><span class="n">input_name</span><span class="p">:</span> <span class="n">X_test</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="n">numpy</span><span class="o">.</span><span class="n">float32</span><span class="p">)})[</span><span class="mi">0</span><span class="p">]</span>
|
||
|
||
<span class="kn">import</span> <span class="nn">pprint</span>
|
||
<span class="n">pprint</span><span class="o">.</span><span class="n">pprint</span><span class="p">(</span><span class="n">prob_rt</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="mi">3</span><span class="p">])</span>
|
||
</pre></div>
|
||
</div>
|
||
<p class="sphx-glr-script-out">Out:</p>
|
||
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>[{0: 0.0041598789393901825, 1: 0.8548984527587891, 2: 0.14094172418117523},
|
||
{0: 0.9442282915115356, 1: 0.055770788341760635, 2: 9.78002844931325e-07},
|
||
{0: 0.05173242464661598, 1: 0.888396143913269, 2: 0.05987146869301796}]
|
||
</pre></div>
|
||
</div>
|
||
<p>Let’s benchmark.</p>
|
||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">timeit</span> <span class="kn">import</span> <span class="n">Timer</span>
|
||
|
||
<span class="k">def</span> <span class="nf">speed</span><span class="p">(</span><span class="n">inst</span><span class="p">,</span> <span class="n">number</span><span class="o">=</span><span class="mi">10</span><span class="p">,</span> <span class="n">repeat</span><span class="o">=</span><span class="mi">20</span><span class="p">):</span>
|
||
<span class="n">timer</span> <span class="o">=</span> <span class="n">Timer</span><span class="p">(</span><span class="n">inst</span><span class="p">,</span> <span class="nb">globals</span><span class="o">=</span><span class="nb">globals</span><span class="p">())</span>
|
||
<span class="n">raw</span> <span class="o">=</span> <span class="n">numpy</span><span class="o">.</span><span class="n">array</span><span class="p">(</span><span class="n">timer</span><span class="o">.</span><span class="n">repeat</span><span class="p">(</span><span class="n">repeat</span><span class="p">,</span> <span class="n">number</span><span class="o">=</span><span class="n">number</span><span class="p">))</span>
|
||
<span class="n">ave</span> <span class="o">=</span> <span class="n">raw</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span> <span class="o">/</span> <span class="nb">len</span><span class="p">(</span><span class="n">raw</span><span class="p">)</span> <span class="o">/</span> <span class="n">number</span>
|
||
<span class="n">mi</span><span class="p">,</span> <span class="n">ma</span> <span class="o">=</span> <span class="n">raw</span><span class="o">.</span><span class="n">min</span><span class="p">()</span> <span class="o">/</span> <span class="n">number</span><span class="p">,</span> <span class="n">raw</span><span class="o">.</span><span class="n">max</span><span class="p">()</span> <span class="o">/</span> <span class="n">number</span>
|
||
<span class="nb">print</span><span class="p">(</span><span class="s2">"Average </span><span class="si">%1.3g</span><span class="s2"> min=</span><span class="si">%1.3g</span><span class="s2"> max=</span><span class="si">%1.3g</span><span class="s2">"</span> <span class="o">%</span> <span class="p">(</span><span class="n">ave</span><span class="p">,</span> <span class="n">mi</span><span class="p">,</span> <span class="n">ma</span><span class="p">))</span>
|
||
<span class="k">return</span> <span class="n">ave</span>
|
||
|
||
<span class="nb">print</span><span class="p">(</span><span class="s2">"Execution time for clr.predict"</span><span class="p">)</span>
|
||
<span class="n">speed</span><span class="p">(</span><span class="s2">"clr.predict(X_test)"</span><span class="p">)</span>
|
||
|
||
<span class="nb">print</span><span class="p">(</span><span class="s2">"Execution time for ONNX Runtime"</span><span class="p">)</span>
|
||
<span class="n">speed</span><span class="p">(</span><span class="s2">"sess.run([label_name], {input_name: X_test.astype(numpy.float32)})[0]"</span><span class="p">)</span>
|
||
</pre></div>
|
||
</div>
|
||
<p class="sphx-glr-script-out">Out:</p>
|
||
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>Execution time for clr.predict
|
||
Average 8.56e-05 min=7.72e-05 max=0.000101
|
||
Execution time for ONNX Runtime
|
||
Average 5.02e-05 min=4.61e-05 max=6.17e-05
|
||
|
||
5.0215695519000296e-05
|
||
</pre></div>
|
||
</div>
|
||
<p>Let’s benchmark a scenario similar to what a webservice
|
||
experiences: the model has to do one prediction at a time
|
||
as opposed to a batch of prediction.</p>
|
||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">loop</span><span class="p">(</span><span class="n">X_test</span><span class="p">,</span> <span class="n">fct</span><span class="p">,</span> <span class="n">n</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
|
||
<span class="n">nrow</span> <span class="o">=</span> <span class="n">X_test</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
|
||
<span class="k">if</span> <span class="n">n</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
|
||
<span class="n">n</span> <span class="o">=</span> <span class="n">nrow</span>
|
||
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="n">n</span><span class="p">):</span>
|
||
<span class="n">im</span> <span class="o">=</span> <span class="n">i</span> <span class="o">%</span> <span class="n">nrow</span>
|
||
<span class="n">fct</span><span class="p">(</span><span class="n">X_test</span><span class="p">[</span><span class="n">im</span><span class="p">:</span> <span class="n">im</span><span class="o">+</span><span class="mi">1</span><span class="p">])</span>
|
||
|
||
<span class="nb">print</span><span class="p">(</span><span class="s2">"Execution time for clr.predict"</span><span class="p">)</span>
|
||
<span class="n">speed</span><span class="p">(</span><span class="s2">"loop(X_test, clr.predict, 100)"</span><span class="p">)</span>
|
||
|
||
<span class="k">def</span> <span class="nf">sess_predict</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
|
||
<span class="k">return</span> <span class="n">sess</span><span class="o">.</span><span class="n">run</span><span class="p">([</span><span class="n">label_name</span><span class="p">],</span> <span class="p">{</span><span class="n">input_name</span><span class="p">:</span> <span class="n">x</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="n">numpy</span><span class="o">.</span><span class="n">float32</span><span class="p">)})[</span><span class="mi">0</span><span class="p">]</span>
|
||
|
||
<span class="nb">print</span><span class="p">(</span><span class="s2">"Execution time for sess_predict"</span><span class="p">)</span>
|
||
<span class="n">speed</span><span class="p">(</span><span class="s2">"loop(X_test, sess_predict, 100)"</span><span class="p">)</span>
|
||
</pre></div>
|
||
</div>
|
||
<p class="sphx-glr-script-out">Out:</p>
|
||
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>Execution time for clr.predict
|
||
Average 0.00723 min=0.00694 max=0.00837
|
||
Execution time for sess_predict
|
||
Average 0.00156 min=0.00152 max=0.00166
|
||
|
||
0.0015644603711552918
|
||
</pre></div>
|
||
</div>
|
||
<p>Let’s do the same for the probabilities.</p>
|
||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="nb">print</span><span class="p">(</span><span class="s2">"Execution time for predict_proba"</span><span class="p">)</span>
|
||
<span class="n">speed</span><span class="p">(</span><span class="s2">"loop(X_test, clr.predict_proba, 100)"</span><span class="p">)</span>
|
||
|
||
<span class="k">def</span> <span class="nf">sess_predict_proba</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
|
||
<span class="k">return</span> <span class="n">sess</span><span class="o">.</span><span class="n">run</span><span class="p">([</span><span class="n">prob_name</span><span class="p">],</span> <span class="p">{</span><span class="n">input_name</span><span class="p">:</span> <span class="n">x</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="n">numpy</span><span class="o">.</span><span class="n">float32</span><span class="p">)})[</span><span class="mi">0</span><span class="p">]</span>
|
||
|
||
<span class="nb">print</span><span class="p">(</span><span class="s2">"Execution time for sess_predict_proba"</span><span class="p">)</span>
|
||
<span class="n">speed</span><span class="p">(</span><span class="s2">"loop(X_test, sess_predict_proba, 100)"</span><span class="p">)</span>
|
||
</pre></div>
|
||
</div>
|
||
<p class="sphx-glr-script-out">Out:</p>
|
||
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>Execution time for predict_proba
|
||
Average 0.0108 min=0.0104 max=0.0115
|
||
Execution time for sess_predict_proba
|
||
Average 0.0017 min=0.00163 max=0.00188
|
||
|
||
0.0016972313076257703
|
||
</pre></div>
|
||
</div>
|
||
<p>This second comparison is better as
|
||
ONNX Runtime, in this experience,
|
||
computes the label and the probabilities
|
||
in every case.</p>
|
||
</div>
|
||
<div class="section" id="benchmark-with-randomforest">
|
||
<h2><a class="toc-backref" href="#id4">Benchmark with RandomForest</a><a class="headerlink" href="#benchmark-with-randomforest" title="Permalink to this headline">¶</a></h2>
|
||
<p>We first train and save a model in ONNX format.</p>
|
||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">sklearn.ensemble</span> <span class="kn">import</span> <span class="n">RandomForestClassifier</span>
|
||
<span class="n">rf</span> <span class="o">=</span> <span class="n">RandomForestClassifier</span><span class="p">()</span>
|
||
<span class="n">rf</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">X_train</span><span class="p">,</span> <span class="n">y_train</span><span class="p">)</span>
|
||
|
||
<span class="n">initial_type</span> <span class="o">=</span> <span class="p">[(</span><span class="s1">'float_input'</span><span class="p">,</span> <span class="n">FloatTensorType</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">4</span><span class="p">]))]</span>
|
||
<span class="n">onx</span> <span class="o">=</span> <span class="n">convert_sklearn</span><span class="p">(</span><span class="n">rf</span><span class="p">,</span> <span class="n">initial_types</span><span class="o">=</span><span class="n">initial_type</span><span class="p">)</span>
|
||
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s2">"rf_iris.onnx"</span><span class="p">,</span> <span class="s2">"wb"</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
|
||
<span class="n">f</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">onx</span><span class="o">.</span><span class="n">SerializeToString</span><span class="p">())</span>
|
||
</pre></div>
|
||
</div>
|
||
<p>We compare.</p>
|
||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">sess</span> <span class="o">=</span> <span class="n">rt</span><span class="o">.</span><span class="n">InferenceSession</span><span class="p">(</span><span class="s2">"rf_iris.onnx"</span><span class="p">)</span>
|
||
|
||
<span class="k">def</span> <span class="nf">sess_predict_proba_rf</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
|
||
<span class="k">return</span> <span class="n">sess</span><span class="o">.</span><span class="n">run</span><span class="p">([</span><span class="n">prob_name</span><span class="p">],</span> <span class="p">{</span><span class="n">input_name</span><span class="p">:</span> <span class="n">x</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="n">numpy</span><span class="o">.</span><span class="n">float32</span><span class="p">)})[</span><span class="mi">0</span><span class="p">]</span>
|
||
|
||
<span class="nb">print</span><span class="p">(</span><span class="s2">"Execution time for predict_proba"</span><span class="p">)</span>
|
||
<span class="n">speed</span><span class="p">(</span><span class="s2">"loop(X_test, rf.predict_proba, 100)"</span><span class="p">)</span>
|
||
|
||
<span class="nb">print</span><span class="p">(</span><span class="s2">"Execution time for sess_predict_proba"</span><span class="p">)</span>
|
||
<span class="n">speed</span><span class="p">(</span><span class="s2">"loop(X_test, sess_predict_proba_rf, 100)"</span><span class="p">)</span>
|
||
</pre></div>
|
||
</div>
|
||
<p class="sphx-glr-script-out">Out:</p>
|
||
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>Execution time for predict_proba
|
||
Average 1.25 min=1.23 max=1.26
|
||
Execution time for sess_predict_proba
|
||
Average 0.00274 min=0.00245 max=0.00442
|
||
|
||
0.0027408220106735826
|
||
</pre></div>
|
||
</div>
|
||
<p>Let’s see with different number of trees.</p>
|
||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">measures</span> <span class="o">=</span> <span class="p">[]</span>
|
||
|
||
<span class="k">for</span> <span class="n">n_trees</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="mi">51</span><span class="p">,</span> <span class="mi">5</span><span class="p">):</span>
|
||
<span class="nb">print</span><span class="p">(</span><span class="n">n_trees</span><span class="p">)</span>
|
||
<span class="n">rf</span> <span class="o">=</span> <span class="n">RandomForestClassifier</span><span class="p">(</span><span class="n">n_estimators</span><span class="o">=</span><span class="n">n_trees</span><span class="p">)</span>
|
||
<span class="n">rf</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">X_train</span><span class="p">,</span> <span class="n">y_train</span><span class="p">)</span>
|
||
<span class="n">initial_type</span> <span class="o">=</span> <span class="p">[(</span><span class="s1">'float_input'</span><span class="p">,</span> <span class="n">FloatTensorType</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">4</span><span class="p">]))]</span>
|
||
<span class="n">onx</span> <span class="o">=</span> <span class="n">convert_sklearn</span><span class="p">(</span><span class="n">rf</span><span class="p">,</span> <span class="n">initial_types</span><span class="o">=</span><span class="n">initial_type</span><span class="p">)</span>
|
||
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s2">"rf_iris_</span><span class="si">%d</span><span class="s2">.onnx"</span> <span class="o">%</span> <span class="n">n_trees</span><span class="p">,</span> <span class="s2">"wb"</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
|
||
<span class="n">f</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">onx</span><span class="o">.</span><span class="n">SerializeToString</span><span class="p">())</span>
|
||
<span class="n">sess</span> <span class="o">=</span> <span class="n">rt</span><span class="o">.</span><span class="n">InferenceSession</span><span class="p">(</span><span class="s2">"rf_iris_</span><span class="si">%d</span><span class="s2">.onnx"</span> <span class="o">%</span> <span class="n">n_trees</span><span class="p">)</span>
|
||
<span class="k">def</span> <span class="nf">sess_predict_proba_loop</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
|
||
<span class="k">return</span> <span class="n">sess</span><span class="o">.</span><span class="n">run</span><span class="p">([</span><span class="n">prob_name</span><span class="p">],</span> <span class="p">{</span><span class="n">input_name</span><span class="p">:</span> <span class="n">x</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="n">numpy</span><span class="o">.</span><span class="n">float32</span><span class="p">)})[</span><span class="mi">0</span><span class="p">]</span>
|
||
<span class="n">tsk</span> <span class="o">=</span> <span class="n">speed</span><span class="p">(</span><span class="s2">"loop(X_test, rf.predict_proba, 100)"</span><span class="p">,</span> <span class="n">number</span><span class="o">=</span><span class="mi">5</span><span class="p">,</span> <span class="n">repeat</span><span class="o">=</span><span class="mi">5</span><span class="p">)</span>
|
||
<span class="n">trt</span> <span class="o">=</span> <span class="n">speed</span><span class="p">(</span><span class="s2">"loop(X_test, sess_predict_proba_loop, 100)"</span><span class="p">,</span> <span class="n">number</span><span class="o">=</span><span class="mi">5</span><span class="p">,</span> <span class="n">repeat</span><span class="o">=</span><span class="mi">5</span><span class="p">)</span>
|
||
<span class="n">measures</span><span class="o">.</span><span class="n">append</span><span class="p">({</span><span class="s1">'n_trees'</span><span class="p">:</span> <span class="n">n_trees</span><span class="p">,</span> <span class="s1">'sklearn'</span><span class="p">:</span> <span class="n">tsk</span><span class="p">,</span> <span class="s1">'rt'</span><span class="p">:</span> <span class="n">trt</span><span class="p">})</span>
|
||
|
||
<span class="kn">from</span> <span class="nn">pandas</span> <span class="kn">import</span> <span class="n">DataFrame</span>
|
||
<span class="n">df</span> <span class="o">=</span> <span class="n">DataFrame</span><span class="p">(</span><span class="n">measures</span><span class="p">)</span>
|
||
<span class="n">ax</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">x</span><span class="o">=</span><span class="s2">"n_trees"</span><span class="p">,</span> <span class="n">y</span><span class="o">=</span><span class="s2">"sklearn"</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s2">"scikit-learn"</span><span class="p">,</span> <span class="n">c</span><span class="o">=</span><span class="s2">"blue"</span><span class="p">,</span> <span class="n">logy</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
|
||
<span class="n">df</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">x</span><span class="o">=</span><span class="s2">"n_trees"</span><span class="p">,</span> <span class="n">y</span><span class="o">=</span><span class="s2">"rt"</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s2">"onnxruntime"</span><span class="p">,</span>
|
||
<span class="n">ax</span><span class="o">=</span><span class="n">ax</span><span class="p">,</span> <span class="n">c</span><span class="o">=</span><span class="s2">"green"</span><span class="p">,</span> <span class="n">logy</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
|
||
<span class="n">ax</span><span class="o">.</span><span class="n">set_xlabel</span><span class="p">(</span><span class="s2">"Number of trees"</span><span class="p">)</span>
|
||
<span class="n">ax</span><span class="o">.</span><span class="n">set_ylabel</span><span class="p">(</span><span class="s2">"Prediction time (s)"</span><span class="p">)</span>
|
||
<span class="n">ax</span><span class="o">.</span><span class="n">set_title</span><span class="p">(</span><span class="s2">"Speed comparison between scikit-learn and ONNX Runtime</span><span class="se">\n</span><span class="s2">For a random forest on Iris dataset"</span><span class="p">)</span>
|
||
<span class="n">ax</span><span class="o">.</span><span class="n">legend</span><span class="p">()</span>
|
||
</pre></div>
|
||
</div>
|
||
<img alt="Speed comparison between scikit-learn and ONNX Runtime For a random forest on Iris dataset" class="sphx-glr-single-img" src="../images/sphx_glr_plot_train_convert_predict_001.png" />
|
||
<p class="sphx-glr-script-out">Out:</p>
|
||
<div class="sphx-glr-script-out highlight-none notranslate"><div class="highlight"><pre><span></span>5
|
||
Average 0.11 min=0.107 max=0.118
|
||
Average 0.00169 min=0.00161 max=0.00177
|
||
10
|
||
Average 0.167 min=0.165 max=0.169
|
||
Average 0.0017 min=0.00165 max=0.00179
|
||
15
|
||
Average 0.228 min=0.227 max=0.231
|
||
Average 0.0017 min=0.00167 max=0.00172
|
||
20
|
||
Average 0.291 min=0.286 max=0.296
|
||
Average 0.00175 min=0.00173 max=0.00176
|
||
25
|
||
Average 0.346 min=0.342 max=0.35
|
||
Average 0.00174 min=0.00172 max=0.00181
|
||
30
|
||
Average 0.407 min=0.404 max=0.41
|
||
Average 0.0018 min=0.00174 max=0.0019
|
||
35
|
||
Average 0.463 min=0.459 max=0.467
|
||
Average 0.0018 min=0.00176 max=0.00187
|
||
40
|
||
Average 0.531 min=0.521 max=0.556
|
||
Average 0.0018 min=0.00179 max=0.00183
|
||
45
|
||
Average 0.582 min=0.577 max=0.597
|
||
Average 0.00187 min=0.00185 max=0.00189
|
||
50
|
||
Average 0.642 min=0.64 max=0.645
|
||
Average 0.00188 min=0.00185 max=0.00196
|
||
|
||
<matplotlib.legend.Legend object at 0x7fca300dcda0>
|
||
</pre></div>
|
||
</div>
|
||
<p class="sphx-glr-timing"><strong>Total running time of the script:</strong> ( 5 minutes 52.417 seconds)</p>
|
||
<div class="sphx-glr-footer class sphx-glr-footer-example docutils container" id="sphx-glr-download-auto-examples-plot-train-convert-predict-py">
|
||
<div class="sphx-glr-download sphx-glr-download-python docutils container">
|
||
<p><a class="reference download internal" download="" href="../downloads/c647c128e0cf2b3db04ce60b41ef1a14/plot_train_convert_predict.py"><code class="xref download docutils literal notranslate"><span class="pre">Download</span> <span class="pre">Python</span> <span class="pre">source</span> <span class="pre">code:</span> <span class="pre">plot_train_convert_predict.py</span></code></a></p>
|
||
</div>
|
||
<div class="sphx-glr-download sphx-glr-download-jupyter docutils container">
|
||
<p><a class="reference download internal" download="" href="../downloads/1680115d3d937dfbb2d86adb705d9c5d/plot_train_convert_predict.ipynb"><code class="xref download docutils literal notranslate"><span class="pre">Download</span> <span class="pre">Jupyter</span> <span class="pre">notebook:</span> <span class="pre">plot_train_convert_predict.ipynb</span></code></a></p>
|
||
</div>
|
||
</div>
|
||
<p class="sphx-glr-signature"><a class="reference external" href="https://sphinx-gallery.github.io">Gallery generated by Sphinx-Gallery</a></p>
|
||
</div>
|
||
</div>
|
||
|
||
|
||
</div>
|
||
|
||
</div>
|
||
</div>
|
||
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
|
||
<div class="sphinxsidebarwrapper">
|
||
<p class="logo"><a href="../index.html">
|
||
<img class="logo" src="../static/ONNX_Runtime_icon.png" alt="Logo"/>
|
||
</a></p>
|
||
<h1 class="logo"><a href="../index.html">ONNX Runtime</a></h1>
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
<h3>Navigation</h3>
|
||
<ul>
|
||
<li class="toctree-l1"><a class="reference internal" href="../tutorial.html">Tutorial</a></li>
|
||
<li class="toctree-l1"><a class="reference internal" href="../api_summary.html">API Summary</a></li>
|
||
<li class="toctree-l1"><a class="reference internal" href="index.html">Gallery of examples</a></li>
|
||
</ul>
|
||
|
||
<div class="relations">
|
||
<h3>Related Topics</h3>
|
||
<ul>
|
||
<li><a href="../index.html">Documentation overview</a><ul>
|
||
<li><a href="index.html">Gallery of examples</a><ul>
|
||
<li>Previous: <a href="plot_common_errors.html" title="previous chapter">Common errors with onnxruntime</a></li>
|
||
</ul></li>
|
||
</ul></li>
|
||
</ul>
|
||
</div>
|
||
<div id="searchbox" style="display: none" role="search">
|
||
<h3 id="searchlabel">Quick search</h3>
|
||
<div class="searchformwrapper">
|
||
<form class="search" action="../search.html" method="get">
|
||
<input type="text" name="q" aria-labelledby="searchlabel" />
|
||
<input type="submit" value="Go" />
|
||
</form>
|
||
</div>
|
||
</div>
|
||
<script>$('#searchbox').show(0);</script>
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
</div>
|
||
</div>
|
||
<div class="clearer"></div>
|
||
</div>
|
||
<div class="footer">
|
||
©2018-2021, Microsoft.
|
||
|
||
|
|
||
Powered by <a href="http://sphinx-doc.org/">Sphinx 3.5.1</a>
|
||
& <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.12</a>
|
||
|
||
|
|
||
<a href="../sources/auto_examples/plot_train_convert_predict.rst.txt"
|
||
rel="nofollow">Page source</a>
|
||
</div>
|
||
|
||
|
||
|
||
|
||
</body>
|
||
</html> |