saymrwulf/onnxruntime
1
0
Fork 0
mirror of https://github.com/saymrwulf/onnxruntime.git synced 2026-07-17 18:40:28 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator
164 commits 2856 branches 57 tags 1.3 GiB
C++ 87.3%
Python 3.1%
C 2.4%
PureBasic 2%
C# 1%
Other 3.7%
Find a file
Xavier Dupré 2d85714183 First version of the documentation (#312) 
* clear branch

* First version of the documentation on github
2019-01-11 11:08:33 -08:00
_downloads First version of the documentation (#312) 2019-01-11 11:08:33 -08:00
_images First version of the documentation (#312) 2019-01-11 11:08:33 -08:00
_modules First version of the documentation (#312) 2019-01-11 11:08:33 -08:00
_sources First version of the documentation (#312) 2019-01-11 11:08:33 -08:00
_static First version of the documentation (#312) 2019-01-11 11:08:33 -08:00
auto_examples First version of the documentation (#312) 2019-01-11 11:08:33 -08:00
.buildinfo First version of the documentation (#312) 2019-01-11 11:08:33 -08:00
.nojekyll First version of the documentation (#312) 2019-01-11 11:08:33 -08:00
api_summary.html First version of the documentation (#312) 2019-01-11 11:08:33 -08:00
examples_md.html First version of the documentation (#312) 2019-01-11 11:08:33 -08:00
genindex.html First version of the documentation (#312) 2019-01-11 11:08:33 -08:00
index.html First version of the documentation (#312) 2019-01-11 11:08:33 -08:00
objects.inv First version of the documentation (#312) 2019-01-11 11:08:33 -08:00
README.html First version of the documentation (#312) 2019-01-11 11:08:33 -08:00
search.html First version of the documentation (#312) 2019-01-11 11:08:33 -08:00
searchindex.js First version of the documentation (#312) 2019-01-11 11:08:33 -08:00
tutorial.html First version of the documentation (#312) 2019-01-11 11:08:33 -08:00

README.html

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
  <meta http-equiv="x-ua-compatible" content="ie=edge">

  <title>ONNX Runtime</title>

  <link rel="stylesheet" href="_static/sphinx-modern-theme.css" type="text/css"/>
  <link rel="stylesheet" href="_static/pygments.css" type="text/css" />
  <link rel="stylesheet" href="_static/gallery.css" type="text/css" />
</head>
<body>
<div class="container">
  <div class="row" style="margin-top: 1rem;">
    <div id="sidebar" class="col-xs-12 col-sm-3">
      <a href="index.html">
        <img style="margin-bottom: 0.5rem;" class="img-fluid" src="_static/ONNX_Runtime_icon.png"/>
      </a>
      <div id="searchbox" style="display: none" role="search">
        <form class="form-inline" action="search.html" method="get">
          <div class="form-group">
            <label class="sr-only" for="searchInput">Search</label>
            <input type="text" class="form-control" name="q" id="searchInput" placeholder="Search">
          </div>
          <button type="submit" class="btn btn-secondary" style="display:none">Go</button>
          <input type="hidden" name="check_keywords" value="yes"/>
          <input type="hidden" name="area" value="default"/>
        </form>
      </div>

      <hr>

        <div id="toc">
        <ul>
<li class="toctree-l1"><a class="reference internal" href="tutorial.html">Tutorial</a><ul>
<li class="toctree-l2"><a class="reference internal" href="tutorial.html#step-1-train-a-model-using-your-favorite-framework">Step 1: Train a model using your favorite framework</a></li>
<li class="toctree-l2"><a class="reference internal" href="tutorial.html#step-2-convert-or-export-the-model-into-onnx-format">Step 2: Convert or export the model into ONNX format</a></li>
<li class="toctree-l2"><a class="reference internal" href="tutorial.html#step-3-load-and-run-the-model-using-onnx-runtime">Step 3: Load and run the model using ONNX Runtime</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="api_summary.html">API Summary</a><ul>
<li class="toctree-l2"><a class="reference internal" href="api_summary.html#device">Device</a></li>
<li class="toctree-l2"><a class="reference internal" href="api_summary.html#examples-and-datasets">Examples and datasets</a></li>
<li class="toctree-l2"><a class="reference internal" href="api_summary.html#load-and-run-a-model">Load and run a model</a></li>
<li class="toctree-l2"><a class="reference internal" href="api_summary.html#backend">Backend</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="auto_examples/index.html">Gallery of examples</a><ul>
<li class="toctree-l2"><a class="reference internal" href="auto_examples/plot_backend.html">ONNX Runtime Backend for ONNX</a></li>
<li class="toctree-l2"><a class="reference internal" href="auto_examples/plot_pipeline.html">Draw a pipeline</a></li>
<li class="toctree-l2"><a class="reference internal" href="auto_examples/plot_load_and_predict.html">Load and predict with ONNX Runtime and a very simple model</a></li>
<li class="toctree-l2"><a class="reference internal" href="auto_examples/plot_profiling.html">Profile the execution of a simple model</a></li>
<li class="toctree-l2"><a class="reference internal" href="auto_examples/plot_metadata.html">Metadata</a></li>
<li class="toctree-l2"><a class="reference internal" href="auto_examples/plot_dl_keras.html">ONNX Runtime for Keras</a></li>
<li class="toctree-l2"><a class="reference internal" href="auto_examples/plot_convert_pipeline_vectorizer.html">Train, convert and predict with ONNX Runtime</a></li>
<li class="toctree-l2"><a class="reference internal" href="auto_examples/plot_common_errors.html">Common errors with onnxruntime</a></li>
<li class="toctree-l2"><a class="reference internal" href="auto_examples/plot_train_convert_predict.html">Train, convert and predict with ONNX Runtime</a></li>
</ul>
</li>
</ul>

        </div>
    </div>
    <div class="col-xs-12 col-sm-9">
      <div class="section" id="onnx-runtime">
<h1>ONNX Runtime<a class="headerlink" href="#onnx-runtime" title="Permalink to this headline">¶</a></h1>
<p>ONNX Runtime
enables high-performance evaluation of trained machine learning (ML)
models while keeping resource usage low.
Building on Microsoft’s dedication to the
<a class="reference external" href="https://onnx.ai/">Open Neural Network Exchange (ONNX)</a>
community, it supports traditional ML models as well
as Deep Learning algorithms in the
<a class="reference external" href="https://github.com/onnx/onnx/blob/master/docs/IR.md">ONNX-ML format</a>.
Documentation is available at
<a class="reference external" href="https://aka.ms/onnxruntime-python">Python Bindings for ONNX Runtime</a>.</p>
<div class="section" id="example">
<h2>Example<a class="headerlink" href="#example" title="Permalink to this headline">¶</a></h2>
<p>The following example demonstrates an end-to-end example
in a very common scenario. A model is trained with <em>scikit-learn</em>
but it has to run very fast in a optimized environment.
The model is then converted into ONNX format and ONNX Runtime
replaces <em>scikit-learn</em> to compute the predictions.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># Train a model.</span>
<span class="kn">from</span> <span class="nn">sklearn.datasets</span> <span class="k">import</span> <span class="n">load_iris</span>
<span class="kn">from</span> <span class="nn">sklearn.model_selection</span> <span class="k">import</span> <span class="n">train_test_split</span>
<span class="kn">from</span> <span class="nn">sklearn.ensemble</span> <span class="k">import</span> <span class="n">RandomForestClassifier</span>
<span class="n">iris</span> <span class="o">=</span> <span class="n">load_iris</span><span class="p">()</span>
<span class="n">X</span><span class="p">,</span> <span class="n">y</span> <span class="o">=</span> <span class="n">iris</span><span class="o">.</span><span class="n">data</span><span class="p">,</span> <span class="n">iris</span><span class="o">.</span><span class="n">target</span>
<span class="n">X_train</span><span class="p">,</span> <span class="n">X_test</span><span class="p">,</span> <span class="n">y_train</span><span class="p">,</span> <span class="n">y_test</span> <span class="o">=</span> <span class="n">train_test_split</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">y</span><span class="p">)</span>
<span class="n">clr</span> <span class="o">=</span> <span class="n">RandomForestClassifier</span><span class="p">()</span>
<span class="n">clr</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">X_train</span><span class="p">,</span> <span class="n">y_train</span><span class="p">)</span>

<span class="c1"># Convert into ONNX format with onnxmltools</span>
<span class="kn">from</span> <span class="nn">skl2onnx</span> <span class="k">import</span> <span class="n">convert_sklearn</span>
<span class="kn">from</span> <span class="nn">skl2onnx.common.data_types</span> <span class="k">import</span> <span class="n">FloatTensorType</span>
<span class="n">initial_type</span> <span class="o">=</span> <span class="p">[(</span><span class="s1">&#39;float_input&#39;</span><span class="p">,</span> <span class="n">FloatTensorType</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">4</span><span class="p">]))]</span>
<span class="n">onx</span> <span class="o">=</span> <span class="n">convert_sklearn</span><span class="p">(</span><span class="n">clr</span><span class="p">,</span> <span class="n">initial_types</span><span class="o">=</span><span class="n">initial_type</span><span class="p">)</span>
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s2">&quot;rf_iris.onnx&quot;</span><span class="p">,</span> <span class="s2">&quot;wb&quot;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
    <span class="n">f</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">onx</span><span class="o">.</span><span class="n">SerializeToString</span><span class="p">())</span>

<span class="c1"># Compute the prediction with ONNX Runtime</span>
<span class="kn">import</span> <span class="nn">onnxruntime</span> <span class="k">as</span> <span class="nn">rt</span>
<span class="kn">import</span> <span class="nn">numpy</span>
<span class="n">sess</span> <span class="o">=</span> <span class="n">rt</span><span class="o">.</span><span class="n">InferenceSession</span><span class="p">(</span><span class="s2">&quot;rf_iris.onnx&quot;</span><span class="p">)</span>
<span class="n">input_name</span> <span class="o">=</span> <span class="n">sess</span><span class="o">.</span><span class="n">get_inputs</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">name</span>
<span class="n">label_name</span> <span class="o">=</span> <span class="n">sess</span><span class="o">.</span><span class="n">get_outputs</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">name</span>
<span class="n">pred_onx</span> <span class="o">=</span> <span class="n">sess</span><span class="o">.</span><span class="n">run</span><span class="p">([</span><span class="n">label_name</span><span class="p">],</span> <span class="p">{</span><span class="n">input_name</span><span class="p">:</span> <span class="n">X_test</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="n">numpy</span><span class="o">.</span><span class="n">float32</span><span class="p">)})[</span><span class="mi">0</span><span class="p">]</span>
</pre></div>
</div>
</div>
<div class="section" id="changes">
<h2>Changes<a class="headerlink" href="#changes" title="Permalink to this headline">¶</a></h2>
<div class="section" id="id1">
<h3>0.1.5<a class="headerlink" href="#id1" title="Permalink to this headline">¶</a></h3>
<p>GA release as part of open sourcing onnxruntime (patch to 0.1.4).</p>
</div>
<div class="section" id="id2">
<h3>0.1.4<a class="headerlink" href="#id2" title="Permalink to this headline">¶</a></h3>
<p>GA release as part of open sourcing onnxruntime.</p>
</div>
<div class="section" id="id3">
<h3>0.1.3<a class="headerlink" href="#id3" title="Permalink to this headline">¶</a></h3>
<p>Fixes a crash on machines which do not support AVX instructions.</p>
</div>
<div class="section" id="id4">
<h3>0.1.2<a class="headerlink" href="#id4" title="Permalink to this headline">¶</a></h3>
<p>First release on Ubuntu 16.04 for CPU and GPU with Cuda 9.1 and Cudnn 7.0,
supports runtime for deep learning models architecture such as AlexNet, ResNet,
XCeption, VGG, Inception, DenseNet, standard linear learner,
standard ensemble learners,
and transform scaler, imputer.</p>
</div>
</div>
</div>

    </div>
  </div>
</div>

<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.0.0/jquery.min.js"
        integrity="sha384-THPy051/pYDQGanwU6poAc/hOdQxjnOEXzbT+OuUAFqNqFjL+4IGLBgCJC3ZOShY"
        crossorigin="anonymous"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/tether/1.2.0/js/tether.min.js"
        integrity="sha384-Plbmg8JY28KFelvJVai01l8WyZzrYWG825m+cZ0eDDS1f7d/js6ikvy1+X+guPIB"
        crossorigin="anonymous"></script>
<script src="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0-alpha.3/js/bootstrap.min.js"
        integrity="sha384-ux8v3A6CPtOTqOzMKiuo3d/DomGaaClxFYdCu2HPMBEkf6x2xiDyJ7gkXU0MWwaD"
        crossorigin="anonymous"></script>
<script src='https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML'></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/lunr.js/0.6.0/lunr.min.js"></script>
<script src="_static/searchtools.js"></script>
<script>$('#searchbox').show(0)</script>
</body>
</html>