2019-01-11 19:08:33 +00:00
<!DOCTYPE html>
2019-12-21 01:11:46 +00:00
< html lang = "en" >
< head >
< meta charset = "utf-8" >
< meta name = "viewport" content = "width=device-width, initial-scale=1, shrink-to-fit=no" >
< meta http-equiv = "x-ua-compatible" content = "ie=edge" >
< title > Tutorial< / title >
< link rel = "stylesheet" href = "_static/alabaster.css" type = "text/css" / >
< link rel = "stylesheet" href = "_static/pygments.css" type = "text/css" / >
< link rel = "stylesheet" href = "_static/gallery.css" type = "text/css" / >
< / head >
< body >
< div class = "container" >
< div class = "row" style = "margin-top: 1rem;" >
< div id = "sidebar" class = "col-xs-12 col-sm-3" >
< a href = "index.html" >
< img style = "margin-bottom: 0.5rem;" class = "img-fluid" src = "_static/ONNX_Runtime_icon.png" / >
< / a >
< div id = "searchbox" style = "display: none" role = "search" >
< form class = "form-inline" action = "search.html" method = "get" >
< div class = "form-group" >
< label class = "sr-only" for = "searchInput" > Search< / label >
< input type = "text" class = "form-control" name = "q" id = "searchInput" placeholder = "Search" >
< / div >
< button type = "submit" class = "btn btn-secondary" style = "display:none" > Go< / button >
< input type = "hidden" name = "check_keywords" value = "yes" / >
< input type = "hidden" name = "area" value = "default" / >
< / form >
< / div >
2019-01-11 19:08:33 +00:00
2019-12-21 01:11:46 +00:00
< hr >
2019-10-30 16:35:16 +00:00
2019-12-21 01:11:46 +00:00
< div id = "toc" >
< ul class = "current" >
< li class = "toctree-l1 current" > < a class = "current reference internal" href = "#" > Tutorial< / a > < ul >
< li class = "toctree-l2" > < a class = "reference internal" href = "#step-1-train-a-model-using-your-favorite-framework" > Step 1: Train a model using your favorite framework< / a > < / li >
< li class = "toctree-l2" > < a class = "reference internal" href = "#step-2-convert-or-export-the-model-into-onnx-format" > Step 2: Convert or export the model into ONNX format< / a > < / li >
< li class = "toctree-l2" > < a class = "reference internal" href = "#step-3-load-and-run-the-model-using-onnx-runtime" > Step 3: Load and run the model using ONNX Runtime< / a > < / li >
< / ul >
< / li >
< li class = "toctree-l1" > < a class = "reference internal" href = "api_summary.html" > API Summary< / a > < ul >
< li class = "toctree-l2" > < a class = "reference internal" href = "api_summary.html#device" > Device< / a > < / li >
< li class = "toctree-l2" > < a class = "reference internal" href = "api_summary.html#examples-and-datasets" > Examples and datasets< / a > < / li >
< li class = "toctree-l2" > < a class = "reference internal" href = "api_summary.html#load-and-run-a-model" > Load and run a model< / a > < / li >
< li class = "toctree-l2" > < a class = "reference internal" href = "api_summary.html#backend" > Backend< / a > < / li >
< / ul >
< / li >
< li class = "toctree-l1" > < a class = "reference internal" href = "auto_examples/index.html" > Gallery of examples< / a > < ul >
< li class = "toctree-l2" > < a class = "reference internal" href = "auto_examples/plot_pipeline.html" > Draw a pipeline< / a > < / li >
< li class = "toctree-l2" > < a class = "reference internal" href = "auto_examples/plot_load_and_predict.html" > Load and predict with ONNX Runtime and a very simple model< / a > < / li >
< li class = "toctree-l2" > < a class = "reference internal" href = "auto_examples/plot_profiling.html" > Profile the execution of a simple model< / a > < / li >
< li class = "toctree-l2" > < a class = "reference internal" href = "auto_examples/plot_backend.html" > ONNX Runtime Backend for ONNX< / a > < / li >
< li class = "toctree-l2" > < a class = "reference internal" href = "auto_examples/plot_metadata.html" > Metadata< / a > < / li >
< li class = "toctree-l2" > < a class = "reference internal" href = "auto_examples/plot_dl_keras.html" > ONNX Runtime for Keras< / a > < / li >
< li class = "toctree-l2" > < a class = "reference internal" href = "auto_examples/plot_convert_pipeline_vectorizer.html" > Train, convert and predict with ONNX Runtime< / a > < / li >
< li class = "toctree-l2" > < a class = "reference internal" href = "auto_examples/plot_common_errors.html" > Common errors with onnxruntime< / a > < / li >
< li class = "toctree-l2" > < a class = "reference internal" href = "auto_examples/plot_train_convert_predict.html" > Train, convert and predict with ONNX Runtime< / a > < / li >
< / ul >
< / li >
< / ul >
2019-12-20 21:38:07 +00:00
2019-12-21 01:11:46 +00:00
< / div >
< / div >
< div class = "col-xs-12 col-sm-9" >
< div class = "section" id = "tutorial" >
2019-01-11 19:08:33 +00:00
< h1 > Tutorial< a class = "headerlink" href = "#tutorial" title = "Permalink to this headline" > ¶< / a > < / h1 >
< p > < em > ONNX Runtime< / em > provides an easy way to run
machine learned models with high performance on CPU or GPU
without dependencies on the training framework.
Machine learning frameworks are usually optimized for
batch training rather than for prediction, which is a
more common scenario in applications, sites, and services.
At a high level, you can:< / p >
< ol class = "arabic simple" >
2019-12-20 21:38:07 +00:00
< li > < p > Train a model using your favorite framework.< / p > < / li >
< li > < p > Convert or export the model into ONNX format.
2019-01-11 19:08:33 +00:00
See < a class = "reference external" href = "https://github.com/onnx/tutorials" > ONNX Tutorials< / a >
2019-12-20 21:38:07 +00:00
for more details.< / p > < / li >
< li > < p > Load and run the model using < em > ONNX Runtime< / em > .< / p > < / li >
2019-01-11 19:08:33 +00:00
< / ol >
< p > In this tutorial, we will briefly create a
pipeline with < em > scikit-learn< / em > , convert it into
ONNX format and run the first predictions.< / p >
< div class = "section" id = "step-1-train-a-model-using-your-favorite-framework" >
2019-12-20 21:38:07 +00:00
< span id = "l-logreg-example" > < / span > < h2 > Step 1: Train a model using your favorite framework< a class = "headerlink" href = "#step-1-train-a-model-using-your-favorite-framework" title = "Permalink to this headline" > ¶< / a > < / h2 >
2019-01-11 19:08:33 +00:00
< p > We’ < / p >
2019-12-20 21:38:07 +00:00
< p > < < < < / p >
< div class = "highlight-default notranslate" > < div class = "highlight" > < pre > < span > < / span > < span class = "kn" > from< / span > < span class = "nn" > sklearn.datasets< / span > < span class = "k" > import< / span > < span class = "n" > load_iris< / span >
< span class = "kn" > from< / span > < span class = "nn" > sklearn.model_selection< / span > < span class = "k" > import< / span > < span class = "n" > train_test_split< / span >
< span class = "n" > iris< / span > < span class = "o" > =< / span > < span class = "n" > load_iris< / span > < span class = "p" > ()< / span >
< span class = "n" > X< / span > < span class = "p" > ,< / span > < span class = "n" > y< / span > < span class = "o" > =< / span > < span class = "n" > iris< / span > < span class = "o" > .< / span > < span class = "n" > data< / span > < span class = "p" > ,< / span > < span class = "n" > iris< / span > < span class = "o" > .< / span > < span class = "n" > target< / span >
< span class = "n" > X_train< / span > < span class = "p" > ,< / span > < span class = "n" > X_test< / span > < span class = "p" > ,< / span > < span class = "n" > y_train< / span > < span class = "p" > ,< / span > < span class = "n" > y_test< / span > < span class = "o" > =< / span > < span class = "n" > train_test_split< / span > < span class = "p" > (< / span > < span class = "n" > X< / span > < span class = "p" > ,< / span > < span class = "n" > y< / span > < span class = "p" > )< / span >
< span class = "kn" > from< / span > < span class = "nn" > sklearn.linear_model< / span > < span class = "k" > import< / span > < span class = "n" > LogisticRegression< / span >
< span class = "n" > clr< / span > < span class = "o" > =< / span > < span class = "n" > LogisticRegression< / span > < span class = "p" > ()< / span >
< span class = "n" > clr< / span > < span class = "o" > .< / span > < span class = "n" > fit< / span > < span class = "p" > (< / span > < span class = "n" > X_train< / span > < span class = "p" > ,< / span > < span class = "n" > y_train< / span > < span class = "p" > )< / span >
< span class = "nb" > print< / span > < span class = "p" > (< / span > < span class = "n" > clr< / span > < span class = "p" > )< / span >
< / pre > < / div >
< / div >
< p > > > > < / p >
2019-12-21 01:11:46 +00:00
< div class = "highlight-default notranslate" > < div class = "highlight" > < pre > < span > < / span > < span class = "n" > LogisticRegression< / span > < span class = "p" > (< / span > < span class = "n" > C< / span > < span class = "o" > =< / span > < span class = "mf" > 1.0< / span > < span class = "p" > ,< / span > < span class = "n" > class_weight< / span > < span class = "o" > =< / span > < span class = "kc" > None< / span > < span class = "p" > ,< / span > < span class = "n" > dual< / span > < span class = "o" > =< / span > < span class = "kc" > False< / span > < span class = "p" > ,< / span > < span class = "n" > fit_intercept< / span > < span class = "o" > =< / span > < span class = "kc" > True< / span > < span class = "p" > ,< / span >
< span class = "n" > intercept_scaling< / span > < span class = "o" > =< / span > < span class = "mi" > 1< / span > < span class = "p" > ,< / span > < span class = "n" > max_iter< / span > < span class = "o" > =< / span > < span class = "mi" > 100< / span > < span class = "p" > ,< / span > < span class = "n" > multi_class< / span > < span class = "o" > =< / span > < span class = "s1" > ' ovr' < / span > < span class = "p" > ,< / span > < span class = "n" > n_jobs< / span > < span class = "o" > =< / span > < span class = "mi" > 1< / span > < span class = "p" > ,< / span >
< span class = "n" > penalty< / span > < span class = "o" > =< / span > < span class = "s1" > ' l2' < / span > < span class = "p" > ,< / span > < span class = "n" > random_state< / span > < span class = "o" > =< / span > < span class = "kc" > None< / span > < span class = "p" > ,< / span > < span class = "n" > solver< / span > < span class = "o" > =< / span > < span class = "s1" > ' liblinear' < / span > < span class = "p" > ,< / span > < span class = "n" > tol< / span > < span class = "o" > =< / span > < span class = "mf" > 0.0001< / span > < span class = "p" > ,< / span >
< span class = "n" > verbose< / span > < span class = "o" > =< / span > < span class = "mi" > 0< / span > < span class = "p" > ,< / span > < span class = "n" > warm_start< / span > < span class = "o" > =< / span > < span class = "kc" > False< / span > < span class = "p" > )< / span >
2019-12-20 21:38:07 +00:00
< / pre > < / div >
< / div >
2019-01-11 19:08:33 +00:00
< / div >
< div class = "section" id = "step-2-convert-or-export-the-model-into-onnx-format" >
< h2 > Step 2: Convert or export the model into ONNX format< a class = "headerlink" href = "#step-2-convert-or-export-the-model-into-onnx-format" title = "Permalink to this headline" > ¶< / a > < / h2 >
< p > < a class = "reference external" href = "https://github.com/onnx/onnx" > ONNX< / a > is a format to describe
the machine learned model.
It defines a set of commonly used operators to compose models.
There are < a class = "reference external" href = "https://github.com/onnx/tutorials" > tools< / a >
to convert other model formats into ONNX. Here we will use
< a class = "reference external" href = "https://github.com/onnx/onnxmltools" > ONNXMLTools< / a > .< / p >
2019-12-20 21:38:07 +00:00
< p > < < < < / p >
< div class = "highlight-default notranslate" > < div class = "highlight" > < pre > < span > < / span > < span class = "kn" > from< / span > < span class = "nn" > skl2onnx< / span > < span class = "k" > import< / span > < span class = "n" > convert_sklearn< / span >
< span class = "kn" > from< / span > < span class = "nn" > skl2onnx.common.data_types< / span > < span class = "k" > import< / span > < span class = "n" > FloatTensorType< / span >
< span class = "n" > initial_type< / span > < span class = "o" > =< / span > < span class = "p" > [(< / span > < span class = "s1" > ' float_input' < / span > < span class = "p" > ,< / span > < span class = "n" > FloatTensorType< / span > < span class = "p" > ([< / span > < span class = "kc" > None< / span > < span class = "p" > ,< / span > < span class = "mi" > 4< / span > < span class = "p" > ]))]< / span >
< span class = "n" > onx< / span > < span class = "o" > =< / span > < span class = "n" > convert_sklearn< / span > < span class = "p" > (< / span > < span class = "n" > clr< / span > < span class = "p" > ,< / span > < span class = "n" > initial_types< / span > < span class = "o" > =< / span > < span class = "n" > initial_type< / span > < span class = "p" > )< / span >
< span class = "k" > with< / span > < span class = "nb" > open< / span > < span class = "p" > (< / span > < span class = "s2" > " logreg_iris.onnx" < / span > < span class = "p" > ,< / span > < span class = "s2" > " wb" < / span > < span class = "p" > )< / span > < span class = "k" > as< / span > < span class = "n" > f< / span > < span class = "p" > :< / span >
< span class = "n" > f< / span > < span class = "o" > .< / span > < span class = "n" > write< / span > < span class = "p" > (< / span > < span class = "n" > onx< / span > < span class = "o" > .< / span > < span class = "n" > SerializeToString< / span > < span class = "p" > ())< / span >
< / pre > < / div >
< / div >
< p > > > > < / p >
< div class = "highlight-default notranslate" > < div class = "highlight" > < pre > < span > < / span >
< / pre > < / div >
< / div >
2019-01-11 19:08:33 +00:00
< / div >
< div class = "section" id = "step-3-load-and-run-the-model-using-onnx-runtime" >
< h2 > Step 3: Load and run the model using ONNX Runtime< a class = "headerlink" href = "#step-3-load-and-run-the-model-using-onnx-runtime" title = "Permalink to this headline" > ¶< / a > < / h2 >
< p > We will use < em > ONNX Runtime< / em > to compute the predictions
for this machine learning model.< / p >
2019-12-20 21:38:07 +00:00
< p > < < < < / p >
< div class = "highlight-default notranslate" > < div class = "highlight" > < pre > < span > < / span > < span class = "kn" > import< / span > < span class = "nn" > numpy< / span >
< span class = "kn" > import< / span > < span class = "nn" > onnxruntime< / span > < span class = "k" > as< / span > < span class = "nn" > rt< / span >
< span class = "n" > sess< / span > < span class = "o" > =< / span > < span class = "n" > rt< / span > < span class = "o" > .< / span > < span class = "n" > InferenceSession< / span > < span class = "p" > (< / span > < span class = "s2" > " logreg_iris.onnx" < / span > < span class = "p" > )< / span >
< span class = "n" > input_name< / span > < span class = "o" > =< / span > < span class = "n" > sess< / span > < span class = "o" > .< / span > < span class = "n" > get_inputs< / span > < span class = "p" > ()[< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > .< / span > < span class = "n" > name< / span >
< span class = "n" > pred_onx< / span > < span class = "o" > =< / span > < span class = "n" > sess< / span > < span class = "o" > .< / span > < span class = "n" > run< / span > < span class = "p" > (< / span > < span class = "kc" > None< / span > < span class = "p" > ,< / span > < span class = "p" > {< / span > < span class = "n" > input_name< / span > < span class = "p" > :< / span > < span class = "n" > X_test< / span > < span class = "o" > .< / span > < span class = "n" > astype< / span > < span class = "p" > (< / span > < span class = "n" > numpy< / span > < span class = "o" > .< / span > < span class = "n" > float32< / span > < span class = "p" > )})[< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span >
< span class = "nb" > print< / span > < span class = "p" > (< / span > < span class = "n" > pred_onx< / span > < span class = "p" > )< / span >
< / pre > < / div >
< / div >
< p > > > > < / p >
2019-12-21 01:11:46 +00:00
< div class = "highlight-default notranslate" > < div class = "highlight" > < pre > < span > < / span > < span class = "p" > [< / span > < span class = "mi" > 1< / span > < span class = "mi" > 1< / span > < span class = "mi" > 2< / span > < span class = "mi" > 2< / span > < span class = "mi" > 0< / span > < span class = "mi" > 1< / span > < span class = "mi" > 2< / span > < span class = "mi" > 2< / span > < span class = "mi" > 2< / span > < span class = "mi" > 0< / span > < span class = "mi" > 0< / span > < span class = "mi" > 1< / span > < span class = "mi" > 2< / span > < span class = "mi" > 1< / span > < span class = "mi" > 1< / span > < span class = "mi" > 2< / span > < span class = "mi" > 0< / span > < span class = "mi" > 1< / span > < span class = "mi" > 0< / span > < span class = "mi" > 0< / span > < span class = "mi" > 0< / span > < span class = "mi" > 2< / span > < span class = "mi" > 0< / span > < span class = "mi" > 0< / span > < span class = "mi" > 1< / span > < span class = "mi" > 1< / span > < span class = "mi" > 2< / span > < span class = "mi" > 0< / span > < span class = "mi" > 0< / span > < span class = "mi" > 0< / span > < span class = "mi" > 2< / span > < span class = "mi" > 1< / span > < span class = "mi" > 1< / span > < span class = "mi" > 1< / span > < span class = "mi" > 1< / span > < span class = "mi" > 2< / span > < span class = "mi" > 0< / span >
2019-12-20 21:38:07 +00:00
< span class = "mi" > 2< / span > < span class = "p" > ]< / span >
< / pre > < / div >
< / div >
2019-01-11 19:08:33 +00:00
< p > The code can be changed to get one specific output
by specifying its name into a list.< / p >
2019-12-20 21:38:07 +00:00
< p > < < < < / p >
< div class = "highlight-default notranslate" > < div class = "highlight" > < pre > < span > < / span > < span class = "kn" > import< / span > < span class = "nn" > numpy< / span >
< span class = "kn" > import< / span > < span class = "nn" > onnxruntime< / span > < span class = "k" > as< / span > < span class = "nn" > rt< / span >
< span class = "n" > sess< / span > < span class = "o" > =< / span > < span class = "n" > rt< / span > < span class = "o" > .< / span > < span class = "n" > InferenceSession< / span > < span class = "p" > (< / span > < span class = "s2" > " logreg_iris.onnx" < / span > < span class = "p" > )< / span >
< span class = "n" > input_name< / span > < span class = "o" > =< / span > < span class = "n" > sess< / span > < span class = "o" > .< / span > < span class = "n" > get_inputs< / span > < span class = "p" > ()[< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > .< / span > < span class = "n" > name< / span >
< span class = "n" > label_name< / span > < span class = "o" > =< / span > < span class = "n" > sess< / span > < span class = "o" > .< / span > < span class = "n" > get_outputs< / span > < span class = "p" > ()[< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span > < span class = "o" > .< / span > < span class = "n" > name< / span >
< span class = "n" > pred_onx< / span > < span class = "o" > =< / span > < span class = "n" > sess< / span > < span class = "o" > .< / span > < span class = "n" > run< / span > < span class = "p" > (< / span >
< span class = "p" > [< / span > < span class = "n" > label_name< / span > < span class = "p" > ],< / span > < span class = "p" > {< / span > < span class = "n" > input_name< / span > < span class = "p" > :< / span > < span class = "n" > X_test< / span > < span class = "o" > .< / span > < span class = "n" > astype< / span > < span class = "p" > (< / span > < span class = "n" > numpy< / span > < span class = "o" > .< / span > < span class = "n" > float32< / span > < span class = "p" > )})[< / span > < span class = "mi" > 0< / span > < span class = "p" > ]< / span >
< span class = "nb" > print< / span > < span class = "p" > (< / span > < span class = "n" > pred_onx< / span > < span class = "p" > )< / span >
< / pre > < / div >
2019-01-11 19:08:33 +00:00
< / div >
2019-12-20 21:38:07 +00:00
< p > > > > < / p >
2019-12-21 01:11:46 +00:00
< div class = "highlight-default notranslate" > < div class = "highlight" > < pre > < span > < / span > < span class = "p" > [< / span > < span class = "mi" > 1< / span > < span class = "mi" > 1< / span > < span class = "mi" > 2< / span > < span class = "mi" > 2< / span > < span class = "mi" > 0< / span > < span class = "mi" > 1< / span > < span class = "mi" > 2< / span > < span class = "mi" > 2< / span > < span class = "mi" > 2< / span > < span class = "mi" > 0< / span > < span class = "mi" > 0< / span > < span class = "mi" > 1< / span > < span class = "mi" > 2< / span > < span class = "mi" > 1< / span > < span class = "mi" > 1< / span > < span class = "mi" > 2< / span > < span class = "mi" > 0< / span > < span class = "mi" > 1< / span > < span class = "mi" > 0< / span > < span class = "mi" > 0< / span > < span class = "mi" > 0< / span > < span class = "mi" > 2< / span > < span class = "mi" > 0< / span > < span class = "mi" > 0< / span > < span class = "mi" > 1< / span > < span class = "mi" > 1< / span > < span class = "mi" > 2< / span > < span class = "mi" > 0< / span > < span class = "mi" > 0< / span > < span class = "mi" > 0< / span > < span class = "mi" > 2< / span > < span class = "mi" > 1< / span > < span class = "mi" > 1< / span > < span class = "mi" > 1< / span > < span class = "mi" > 1< / span > < span class = "mi" > 2< / span > < span class = "mi" > 0< / span >
2019-12-20 21:38:07 +00:00
< span class = "mi" > 2< / span > < span class = "p" > ]< / span >
< / pre > < / div >
2019-01-11 19:08:33 +00:00
< / div >
2019-12-20 21:38:07 +00:00
< / div >
< / div >
2019-01-11 19:08:33 +00:00
< / div >
2019-12-21 01:11:46 +00:00
< / div >
2019-01-11 19:08:33 +00:00
< / div >
2019-12-21 01:11:46 +00:00
< script src = "https://ajax.googleapis.com/ajax/libs/jquery/3.0.0/jquery.min.js"
integrity="sha384-THPy051/pYDQGanwU6poAc/hOdQxjnOEXzbT+OuUAFqNqFjL+4IGLBgCJC3ZOShY"
crossorigin="anonymous">< / script >
< script src = "https://cdnjs.cloudflare.com/ajax/libs/tether/1.2.0/js/tether.min.js"
integrity="sha384-Plbmg8JY28KFelvJVai01l8WyZzrYWG825m+cZ0eDDS1f7d/js6ikvy1+X+guPIB"
crossorigin="anonymous">< / script >
< script src = "https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0-alpha.3/js/bootstrap.min.js"
integrity="sha384-ux8v3A6CPtOTqOzMKiuo3d/DomGaaClxFYdCu2HPMBEkf6x2xiDyJ7gkXU0MWwaD"
crossorigin="anonymous">< / script >
< script src = 'https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML' > < / script >
< script src = "https://cdnjs.cloudflare.com/ajax/libs/lunr.js/0.6.0/lunr.min.js" > < / script >
< script src = "_static/searchtools.js" > < / script >
< script > $ ( '#searchbox' ) . show ( 0 ) < / script >
< / body >
2019-01-11 19:08:33 +00:00
< / html >
