
.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/plot_train_convert_predict.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        Click :ref:`here <sphx_glr_download_auto_examples_plot_train_convert_predict.py>`
        to download the full example code

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_plot_train_convert_predict.py:


.. _l-logreg-example:

Train, convert and predict with ONNX Runtime
============================================

This example demonstrates an end to end scenario
starting with the training of a machine learned model
to its use in its converted from.

.. contents::
    :local:

Train a logistic regression
+++++++++++++++++++++++++++

The first step consists in retrieving the iris datset.

.. GENERATED FROM PYTHON SOURCE LINES 23-33

.. code-block:: default


    from sklearn.datasets import load_iris

    iris = load_iris()
    X, y = iris.data, iris.target

    from sklearn.model_selection import train_test_split

    X_train, X_test, y_train, y_test = train_test_split(X, y)








.. GENERATED FROM PYTHON SOURCE LINES 34-35

Then we fit a model.

.. GENERATED FROM PYTHON SOURCE LINES 35-41

.. code-block:: default


    from sklearn.linear_model import LogisticRegression

    clr = LogisticRegression()
    clr.fit(X_train, y_train)





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    /home/runner/.local/lib/python3.10/site-packages/sklearn/linear_model/_logistic.py:444: ConvergenceWarning: lbfgs failed to converge (status=1):
    STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.

    Increase the number of iterations (max_iter) or scale the data as shown in:
        https://scikit-learn.org/stable/modules/preprocessing.html
    Please also refer to the documentation for alternative solver options:
        https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
      n_iter_i = _check_optimize_result(


.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <style>#sk-container-id-2 {color: black;background-color: white;}#sk-container-id-2 pre{padding: 0;}#sk-container-id-2 div.sk-toggleable {background-color: white;}#sk-container-id-2 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-2 label.sk-toggleable__label-arrow:before {content: "▸";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-2 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-2 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-2 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-2 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-2 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-2 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: "▾";}#sk-container-id-2 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-2 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-2 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-2 div.sk-parallel-item::after {content: "";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-2 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 div.sk-serial::before {content: "";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-2 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-2 div.sk-item {position: relative;z-index: 1;}#sk-container-id-2 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-2 div.sk-item::before, #sk-container-id-2 div.sk-parallel-item::before {content: "";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-2 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-2 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-2 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-2 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-2 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-2 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-2 div.sk-label-container {text-align: center;}#sk-container-id-2 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-2 div.sk-text-repr-fallback {display: none;}</style><div id="sk-container-id-2" class="sk-top-container"><div class="sk-text-repr-fallback"><pre>LogisticRegression()</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class="sk-container" hidden><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="sk-estimator-id-4" type="checkbox" checked><label for="sk-estimator-id-4" class="sk-toggleable__label sk-toggleable__label-arrow">LogisticRegression</label><div class="sk-toggleable__content"><pre>LogisticRegression()</pre></div></div></div></div></div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 42-44

We compute the prediction on the test set
and we show the confusion matrix.

.. GENERATED FROM PYTHON SOURCE LINES 44-49

.. code-block:: default

    from sklearn.metrics import confusion_matrix

    pred = clr.predict(X_test)
    print(confusion_matrix(y_test, pred))





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    [[14  0  0]
     [ 0 11  0]
     [ 0  1 12]]




.. GENERATED FROM PYTHON SOURCE LINES 50-56

Conversion to ONNX format
+++++++++++++++++++++++++

We use module
`sklearn-onnx <https://github.com/onnx/sklearn-onnx>`_
to convert the model into ONNX format.

.. GENERATED FROM PYTHON SOURCE LINES 56-65

.. code-block:: default


    from skl2onnx import convert_sklearn
    from skl2onnx.common.data_types import FloatTensorType

    initial_type = [("float_input", FloatTensorType([None, 4]))]
    onx = convert_sklearn(clr, initial_types=initial_type)
    with open("logreg_iris.onnx", "wb") as f:
        f.write(onx.SerializeToString())








.. GENERATED FROM PYTHON SOURCE LINES 66-68

We load the model with ONNX Runtime and look at
its input and output.

.. GENERATED FROM PYTHON SOURCE LINES 68-76

.. code-block:: default


    import onnxruntime as rt

    sess = rt.InferenceSession("logreg_iris.onnx", providers=rt.get_available_providers())

    print("input name='{}' and shape={}".format(sess.get_inputs()[0].name, sess.get_inputs()[0].shape))
    print("output name='{}' and shape={}".format(sess.get_outputs()[0].name, sess.get_outputs()[0].shape))





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    input name='float_input' and shape=[None, 4]
    output name='output_label' and shape=[None]




.. GENERATED FROM PYTHON SOURCE LINES 77-78

We compute the predictions.

.. GENERATED FROM PYTHON SOURCE LINES 78-87

.. code-block:: default


    input_name = sess.get_inputs()[0].name
    label_name = sess.get_outputs()[0].name

    import numpy

    pred_onx = sess.run([label_name], {input_name: X_test.astype(numpy.float32)})[0]
    print(confusion_matrix(pred, pred_onx))





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    [[14  0  0]
     [ 0 12  0]
     [ 0  0 12]]




.. GENERATED FROM PYTHON SOURCE LINES 88-97

The prediction are perfectly identical.

Probabilities
+++++++++++++

Probabilities are needed to compute other
relevant metrics such as the ROC Curve.
Let's see how to get them first with
scikit-learn.

.. GENERATED FROM PYTHON SOURCE LINES 97-101

.. code-block:: default


    prob_sklearn = clr.predict_proba(X_test)
    print(prob_sklearn[:3])





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    [[1.90908960e-01 8.02874408e-01 6.21663156e-03]
     [2.81698299e-02 9.12759445e-01 5.90707247e-02]
     [9.67801591e-01 3.21983324e-02 7.64754321e-08]]




.. GENERATED FROM PYTHON SOURCE LINES 102-104

And then with ONNX Runtime.
The probabilies appear to be

.. GENERATED FROM PYTHON SOURCE LINES 104-112

.. code-block:: default


    prob_name = sess.get_outputs()[1].name
    prob_rt = sess.run([prob_name], {input_name: X_test.astype(numpy.float32)})[0]

    import pprint

    pprint.pprint(prob_rt[0:3])





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    [{0: 0.19090914726257324, 1: 0.8028742074966431, 2: 0.0062166303396224976},
     {0: 0.02816985361278057, 1: 0.9127594232559204, 2: 0.059070732444524765},
     {0: 0.9678016304969788, 1: 0.03219832107424736, 2: 7.647536648391906e-08}]




.. GENERATED FROM PYTHON SOURCE LINES 113-114

Let's benchmark.

.. GENERATED FROM PYTHON SOURCE LINES 114-132

.. code-block:: default

    from timeit import Timer


    def speed(inst, number=10, repeat=20):
        timer = Timer(inst, globals=globals())
        raw = numpy.array(timer.repeat(repeat, number=number))
        ave = raw.sum() / len(raw) / number
        mi, ma = raw.min() / number, raw.max() / number
        print("Average %1.3g min=%1.3g max=%1.3g" % (ave, mi, ma))
        return ave


    print("Execution time for clr.predict")
    speed("clr.predict(X_test)")

    print("Execution time for ONNX Runtime")
    speed("sess.run([label_name], {input_name: X_test.astype(numpy.float32)})[0]")





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Execution time for clr.predict
    Average 4.44e-05 min=4.24e-05 max=5.4e-05
    Execution time for ONNX Runtime
    Average 2.21e-05 min=2.16e-05 max=2.71e-05

    2.2087500000509407e-05



.. GENERATED FROM PYTHON SOURCE LINES 133-136

Let's benchmark a scenario similar to what a webservice
experiences: the model has to do one prediction at a time
as opposed to a batch of prediction.

.. GENERATED FROM PYTHON SOURCE LINES 136-158

.. code-block:: default



    def loop(X_test, fct, n=None):
        nrow = X_test.shape[0]
        if n is None:
            n = nrow
        for i in range(0, n):
            im = i % nrow
            fct(X_test[im : im + 1])


    print("Execution time for clr.predict")
    speed("loop(X_test, clr.predict, 100)")


    def sess_predict(x):
        return sess.run([label_name], {input_name: x.astype(numpy.float32)})[0]


    print("Execution time for sess_predict")
    speed("loop(X_test, sess_predict, 100)")





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Execution time for clr.predict
    Average 0.00406 min=0.00404 max=0.00414
    Execution time for sess_predict
    Average 0.00104 min=0.00102 max=0.00107

    0.00103513229499967



.. GENERATED FROM PYTHON SOURCE LINES 159-160

Let's do the same for the probabilities.

.. GENERATED FROM PYTHON SOURCE LINES 160-172

.. code-block:: default


    print("Execution time for predict_proba")
    speed("loop(X_test, clr.predict_proba, 100)")


    def sess_predict_proba(x):
        return sess.run([prob_name], {input_name: x.astype(numpy.float32)})[0]


    print("Execution time for sess_predict_proba")
    speed("loop(X_test, sess_predict_proba, 100)")





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Execution time for predict_proba
    Average 0.0061 min=0.00608 max=0.00626
    Execution time for sess_predict_proba
    Average 0.00108 min=0.00107 max=0.0011

    0.001081757285000009



.. GENERATED FROM PYTHON SOURCE LINES 173-177

This second comparison is better as
ONNX Runtime, in this experience,
computes the label and the probabilities
in every case.

.. GENERATED FROM PYTHON SOURCE LINES 179-183

Benchmark with RandomForest
+++++++++++++++++++++++++++

We first train and save a model in ONNX format.

.. GENERATED FROM PYTHON SOURCE LINES 183-193

.. code-block:: default

    from sklearn.ensemble import RandomForestClassifier

    rf = RandomForestClassifier()
    rf.fit(X_train, y_train)

    initial_type = [("float_input", FloatTensorType([1, 4]))]
    onx = convert_sklearn(rf, initial_types=initial_type)
    with open("rf_iris.onnx", "wb") as f:
        f.write(onx.SerializeToString())








.. GENERATED FROM PYTHON SOURCE LINES 194-195

We compare.

.. GENERATED FROM PYTHON SOURCE LINES 195-209

.. code-block:: default


    sess = rt.InferenceSession("rf_iris.onnx", providers=rt.get_available_providers())


    def sess_predict_proba_rf(x):
        return sess.run([prob_name], {input_name: x.astype(numpy.float32)})[0]


    print("Execution time for predict_proba")
    speed("loop(X_test, rf.predict_proba, 100)")

    print("Execution time for sess_predict_proba")
    speed("loop(X_test, sess_predict_proba_rf, 100)")





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Execution time for predict_proba
    Average 0.674 min=0.672 max=0.678
    Execution time for sess_predict_proba
    Average 0.0013 min=0.00129 max=0.00133

    0.001301839815000676



.. GENERATED FROM PYTHON SOURCE LINES 210-211

Let's see with different number of trees.

.. GENERATED FROM PYTHON SOURCE LINES 211-240

.. code-block:: default


    measures = []

    for n_trees in range(5, 51, 5):
        print(n_trees)
        rf = RandomForestClassifier(n_estimators=n_trees)
        rf.fit(X_train, y_train)
        initial_type = [("float_input", FloatTensorType([1, 4]))]
        onx = convert_sklearn(rf, initial_types=initial_type)
        with open("rf_iris_%d.onnx" % n_trees, "wb") as f:
            f.write(onx.SerializeToString())
        sess = rt.InferenceSession("rf_iris_%d.onnx" % n_trees, providers=rt.get_available_providers())

        def sess_predict_proba_loop(x):
            return sess.run([prob_name], {input_name: x.astype(numpy.float32)})[0]

        tsk = speed("loop(X_test, rf.predict_proba, 100)", number=5, repeat=5)
        trt = speed("loop(X_test, sess_predict_proba_loop, 100)", number=5, repeat=5)
        measures.append({"n_trees": n_trees, "sklearn": tsk, "rt": trt})

    from pandas import DataFrame

    df = DataFrame(measures)
    ax = df.plot(x="n_trees", y="sklearn", label="scikit-learn", c="blue", logy=True)
    df.plot(x="n_trees", y="rt", label="onnxruntime", ax=ax, c="green", logy=True)
    ax.set_xlabel("Number of trees")
    ax.set_ylabel("Prediction time (s)")
    ax.set_title("Speed comparison between scikit-learn and ONNX Runtime\nFor a random forest on Iris dataset")
    ax.legend()



.. image-sg:: /auto_examples/images/sphx_glr_plot_train_convert_predict_001.png
   :alt: Speed comparison between scikit-learn and ONNX Runtime For a random forest on Iris dataset
   :srcset: /auto_examples/images/sphx_glr_plot_train_convert_predict_001.png
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    5
    Average 0.0491 min=0.049 max=0.0491
    Average 0.00103 min=0.00102 max=0.00105
    10
    Average 0.0823 min=0.0822 max=0.0824
    Average 0.00104 min=0.00102 max=0.00106
    15
    Average 0.115 min=0.115 max=0.115
    Average 0.00103 min=0.00102 max=0.00106
    20
    Average 0.148 min=0.148 max=0.148
    Average 0.00105 min=0.00104 max=0.00108
    25
    Average 0.181 min=0.181 max=0.182
    Average 0.00107 min=0.00106 max=0.00109
    30
    Average 0.214 min=0.214 max=0.214
    Average 0.00107 min=0.00106 max=0.00109
    35
    Average 0.247 min=0.247 max=0.248
    Average 0.00109 min=0.00108 max=0.00111
    40
    Average 0.28 min=0.279 max=0.28
    Average 0.00109 min=0.00108 max=0.00112
    45
    Average 0.312 min=0.311 max=0.312
    Average 0.00112 min=0.00111 max=0.00115
    50
    Average 0.345 min=0.344 max=0.345
    Average 0.00112 min=0.00111 max=0.00114

    <matplotlib.legend.Legend object at 0x7f0fd5bd2a40>




.. rst-class:: sphx-glr-timing

   **Total running time of the script:** ( 3 minutes  8.150 seconds)


.. _sphx_glr_download_auto_examples_plot_train_convert_predict.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example


    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_train_convert_predict.py <plot_train_convert_predict.py>`

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_train_convert_predict.ipynb <plot_train_convert_predict.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_
