saymrwulf/onnxruntime
1
0
Fork 0
mirror of https://github.com/saymrwulf/onnxruntime.git synced 2026-07-16 18:31:27 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
onnxruntime/python/_modules/onnxruntime/capi/session.html
Xavier Dupré 99b2c8a053
Update documentation (#3380) 
Co-authored-by: xavier dupré <xavier.dupre@gmail.com>
2020-04-06 15:52:48 -07:00

270 lines
No EOL
26 KiB
HTML
Raw Blame History

<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="en">
<head>
<meta charset="utf-8" />
<title>onnxruntime.capi.session &#8212; ONNX Runtime 1.2.0 documentation</title>
<link rel="stylesheet" href="../../../_static/alabaster.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" type="text/css" href="../../../_static/gallery.css" />
<link rel="stylesheet" type="text/css" href="../../../_static/graphviz.css" />
<script type="text/javascript" id="documentation_options" data-url_root="../../../" src="../../../_static/documentation_options.js"></script>
<script type="text/javascript" src="../../../_static/jquery.js"></script>
<script type="text/javascript" src="../../../_static/underscore.js"></script>
<script type="text/javascript" src="../../../_static/doctools.js"></script>
<script type="text/javascript" src="../../../_static/language_data.js"></script>
<link rel="index" title="Index" href="../../../genindex.html" />
<link rel="search" title="Search" href="../../../search.html" />
<link rel="stylesheet" href="../../../_static/custom.css" type="text/css" />
<meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
</head><body>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<h1>Source code for onnxruntime.capi.session</h1><div class="highlight"><pre>
<span></span><span class="c1">#-------------------------------------------------------------------------</span>
<span class="c1"># Copyright (c) Microsoft Corporation. All rights reserved.</span>
<span class="c1"># Licensed under the MIT License.</span>
<span class="c1">#--------------------------------------------------------------------------</span>
<span class="kn">import</span> <span class="nn">sys</span>
<span class="kn">import</span> <span class="nn">os</span>
<span class="kn">from</span> <span class="nn">onnxruntime.capi</span> <span class="k">import</span> <span class="n">_pybind_state</span> <span class="k">as</span> <span class="n">C</span>
<div class="viewcode-block" id="InferenceSession"><a class="viewcode-back" href="../../../api_summary.html#onnxruntime.InferenceSession">[docs]</a><span class="k">class</span> <span class="nc">InferenceSession</span><span class="p">:</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd"> This is the main class used to run a model.</span>
<span class="sd"> &quot;&quot;&quot;</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">path_or_bytes</span><span class="p">,</span> <span class="n">sess_options</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">providers</span><span class="o">=</span><span class="p">[]):</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd"> :param path_or_bytes: filename or serialized model in a byte string</span>
<span class="sd"> :param sess_options: session options</span>
<span class="sd"> :param providers: providers to use for session. If empty, will use</span>
<span class="sd"> all available providers.</span>
<span class="sd"> &quot;&quot;&quot;</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_path_or_bytes</span> <span class="o">=</span> <span class="n">path_or_bytes</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_sess_options</span> <span class="o">=</span> <span class="n">sess_options</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_load_model</span><span class="p">(</span><span class="n">providers</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_enable_fallback</span> <span class="o">=</span> <span class="kc">True</span>
<span class="k">def</span> <span class="nf">_load_model</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">providers</span><span class="o">=</span><span class="p">[]):</span>
<span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_path_or_bytes</span><span class="p">,</span> <span class="nb">str</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_sess</span> <span class="o">=</span> <span class="n">C</span><span class="o">.</span><span class="n">InferenceSession</span><span class="p">(</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_sess_options</span> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_sess_options</span> <span class="k">else</span> <span class="n">C</span><span class="o">.</span><span class="n">get_default_session_options</span><span class="p">(),</span> <span class="bp">self</span><span class="o">.</span><span class="n">_path_or_bytes</span><span class="p">,</span>
<span class="kc">True</span><span class="p">)</span>
<span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_path_or_bytes</span><span class="p">,</span> <span class="nb">bytes</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_sess</span> <span class="o">=</span> <span class="n">C</span><span class="o">.</span><span class="n">InferenceSession</span><span class="p">(</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_sess_options</span> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_sess_options</span> <span class="k">else</span> <span class="n">C</span><span class="o">.</span><span class="n">get_default_session_options</span><span class="p">(),</span> <span class="bp">self</span><span class="o">.</span><span class="n">_path_or_bytes</span><span class="p">,</span>
<span class="kc">False</span><span class="p">)</span>
<span class="c1"># elif isinstance(self._path_or_bytes, tuple):</span>
<span class="c1"># to remove, hidden trick</span>
<span class="c1"># self._sess.load_model_no_init(self._path_or_bytes[0], providers)</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">raise</span> <span class="ne">TypeError</span><span class="p">(</span><span class="s2">&quot;Unable to load from type &#39;</span><span class="si">{0}</span><span class="s2">&#39;&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="nb">type</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_path_or_bytes</span><span class="p">)))</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_sess</span><span class="o">.</span><span class="n">load_model</span><span class="p">(</span><span class="n">providers</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_session_options</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_sess</span><span class="o">.</span><span class="n">session_options</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_inputs_meta</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_sess</span><span class="o">.</span><span class="n">inputs_meta</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_outputs_meta</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_sess</span><span class="o">.</span><span class="n">outputs_meta</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_overridable_initializers</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_sess</span><span class="o">.</span><span class="n">overridable_initializers</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_model_meta</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_sess</span><span class="o">.</span><span class="n">model_meta</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_providers</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_sess</span><span class="o">.</span><span class="n">get_providers</span><span class="p">()</span>
<span class="c1"># Tensorrt can fall back to CUDA. All others fall back to CPU.</span>
<span class="k">if</span> <span class="s1">&#39;TensorrtExecutionProvider&#39;</span> <span class="ow">in</span> <span class="n">C</span><span class="o">.</span><span class="n">get_available_providers</span><span class="p">():</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_fallback_providers</span> <span class="o">=</span> <span class="p">[</span><span class="s1">&#39;CUDAExecutionProvider&#39;</span><span class="p">,</span> <span class="s1">&#39;CPUExecutionProvider&#39;</span><span class="p">]</span>
<span class="k">else</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_fallback_providers</span> <span class="o">=</span> <span class="p">[</span><span class="s1">&#39;CPUExecutionProvider&#39;</span><span class="p">]</span>
<span class="k">def</span> <span class="nf">_reset_session</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="s2">&quot;release underlying session object.&quot;</span>
<span class="c1"># meta data references session internal structures</span>
<span class="c1"># so they must be set to None to decrement _sess reference count.</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_inputs_meta</span> <span class="o">=</span> <span class="kc">None</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_outputs_meta</span> <span class="o">=</span> <span class="kc">None</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_overridable_initializers</span> <span class="o">=</span> <span class="kc">None</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_model_meta</span> <span class="o">=</span> <span class="kc">None</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_providers</span> <span class="o">=</span> <span class="kc">None</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_sess</span> <span class="o">=</span> <span class="kc">None</span>
<div class="viewcode-block" id="InferenceSession.get_session_options"><a class="viewcode-back" href="../../../api_summary.html#onnxruntime.InferenceSession.get_session_options">[docs]</a> <span class="k">def</span> <span class="nf">get_session_options</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="s2">&quot;Return the session options. See :class:`onnxruntime.SessionOptions`.&quot;</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_session_options</span></div>
<div class="viewcode-block" id="InferenceSession.get_inputs"><a class="viewcode-back" href="../../../api_summary.html#onnxruntime.InferenceSession.get_inputs">[docs]</a> <span class="k">def</span> <span class="nf">get_inputs</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="s2">&quot;Return the inputs metadata as a list of :class:`onnxruntime.NodeArg`.&quot;</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_inputs_meta</span></div>
<div class="viewcode-block" id="InferenceSession.get_outputs"><a class="viewcode-back" href="../../../api_summary.html#onnxruntime.InferenceSession.get_outputs">[docs]</a> <span class="k">def</span> <span class="nf">get_outputs</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="s2">&quot;Return the outputs metadata as a list of :class:`onnxruntime.NodeArg`.&quot;</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_outputs_meta</span></div>
<div class="viewcode-block" id="InferenceSession.get_overridable_initializers"><a class="viewcode-back" href="../../../api_summary.html#onnxruntime.InferenceSession.get_overridable_initializers">[docs]</a> <span class="k">def</span> <span class="nf">get_overridable_initializers</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="s2">&quot;Return the inputs (including initializers) metadata as a list of :class:`onnxruntime.NodeArg`.&quot;</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_overridable_initializers</span></div>
<div class="viewcode-block" id="InferenceSession.get_modelmeta"><a class="viewcode-back" href="../../../api_summary.html#onnxruntime.InferenceSession.get_modelmeta">[docs]</a> <span class="k">def</span> <span class="nf">get_modelmeta</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="s2">&quot;Return the metadata. See :class:`onnxruntime.ModelMetadata`.&quot;</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_model_meta</span></div>
<div class="viewcode-block" id="InferenceSession.get_providers"><a class="viewcode-back" href="../../../api_summary.html#onnxruntime.InferenceSession.get_providers">[docs]</a> <span class="k">def</span> <span class="nf">get_providers</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="s2">&quot;Return list of registered execution providers.&quot;</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_providers</span></div>
<div class="viewcode-block" id="InferenceSession.set_providers"><a class="viewcode-back" href="../../../api_summary.html#onnxruntime.InferenceSession.set_providers">[docs]</a> <span class="k">def</span> <span class="nf">set_providers</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">providers</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd"> Register the input list of execution providers. The underlying session is re-created.</span>
<span class="sd"> :param providers: list of execution providers</span>
<span class="sd"> The list of providers is ordered by Priority. For example [&#39;CUDAExecutionProvider&#39;, &#39;CPUExecutionProvider&#39;] means</span>
<span class="sd"> execute a node using CUDAExecutionProvider if capable, otherwise execute using CPUExecutionProvider.</span>
<span class="sd"> &quot;&quot;&quot;</span>
<span class="k">if</span> <span class="ow">not</span> <span class="nb">set</span><span class="p">(</span><span class="n">providers</span><span class="p">)</span><span class="o">.</span><span class="n">issubset</span><span class="p">(</span><span class="n">C</span><span class="o">.</span><span class="n">get_available_providers</span><span class="p">()):</span>
<span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="s2">&quot;</span><span class="si">{}</span><span class="s2"> does not contain a subset of available providers </span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span>
<span class="n">providers</span><span class="p">,</span> <span class="n">C</span><span class="o">.</span><span class="n">get_available_providers</span><span class="p">()))</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_reset_session</span><span class="p">()</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_load_model</span><span class="p">(</span><span class="n">providers</span><span class="p">)</span></div>
<div class="viewcode-block" id="InferenceSession.disable_fallback"><a class="viewcode-back" href="../../../api_summary.html#onnxruntime.InferenceSession.disable_fallback">[docs]</a> <span class="k">def</span> <span class="nf">disable_fallback</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd"> Disable session.run() fallback mechanism.</span>
<span class="sd"> &quot;&quot;&quot;</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_enable_fallback</span> <span class="o">=</span> <span class="kc">False</span></div>
<div class="viewcode-block" id="InferenceSession.enable_fallback"><a class="viewcode-back" href="../../../api_summary.html#onnxruntime.InferenceSession.enable_fallback">[docs]</a> <span class="k">def</span> <span class="nf">enable_fallback</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd"> Enable session.Run() fallback mechanism. If session.Run() fails due to an internal Execution Provider failure, reset the Execution Providers</span>
<span class="sd"> enabled for this session.</span>
<span class="sd"> If GPU is enabled, fall back to CUDAExecutionProvider.</span>
<span class="sd"> otherwise fall back to CPUExecutionProvider.</span>
<span class="sd"> &quot;&quot;&quot;</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_enable_fallback</span> <span class="o">=</span> <span class="kc">True</span></div>
<div class="viewcode-block" id="InferenceSession.run"><a class="viewcode-back" href="../../../api_summary.html#onnxruntime.InferenceSession.run">[docs]</a> <span class="k">def</span> <span class="nf">run</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">output_names</span><span class="p">,</span> <span class="n">input_feed</span><span class="p">,</span> <span class="n">run_options</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd"> Compute the predictions.</span>
<span class="sd"> :param output_names: name of the outputs</span>
<span class="sd"> :param input_feed: dictionary ``{ input_name: input_value }``</span>
<span class="sd"> :param run_options: See :class:`onnxruntime.RunOptions`.</span>
<span class="sd"> ::</span>
<span class="sd"> sess.run([output_name], {input_name: x})</span>
<span class="sd"> &quot;&quot;&quot;</span>
<span class="n">num_required_inputs</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_inputs_meta</span><span class="p">)</span>
<span class="n">num_inputs</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">input_feed</span><span class="p">)</span>
<span class="c1"># the graph may have optional inputs used to override initializers. allow for that.</span>
<span class="k">if</span> <span class="n">num_inputs</span> <span class="o">&lt;</span> <span class="n">num_required_inputs</span><span class="p">:</span>
<span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="s2">&quot;Model requires </span><span class="si">{}</span><span class="s2"> inputs. Input Feed contains </span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">num_required_inputs</span><span class="p">,</span> <span class="n">num_inputs</span><span class="p">))</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">output_names</span><span class="p">:</span>
<span class="n">output_names</span> <span class="o">=</span> <span class="p">[</span><span class="n">output</span><span class="o">.</span><span class="n">name</span> <span class="k">for</span> <span class="n">output</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_outputs_meta</span><span class="p">]</span>
<span class="k">try</span><span class="p">:</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_sess</span><span class="o">.</span><span class="n">run</span><span class="p">(</span><span class="n">output_names</span><span class="p">,</span> <span class="n">input_feed</span><span class="p">,</span> <span class="n">run_options</span><span class="p">)</span>
<span class="k">except</span> <span class="n">C</span><span class="o">.</span><span class="n">EPFail</span> <span class="k">as</span> <span class="n">err</span><span class="p">:</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_enable_fallback</span><span class="p">:</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;EP Error: </span><span class="si">{}</span><span class="s2"> using </span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="nb">str</span><span class="p">(</span><span class="n">err</span><span class="p">),</span> <span class="bp">self</span><span class="o">.</span><span class="n">_providers</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Falling back to </span><span class="si">{}</span><span class="s2"> and retrying.&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_fallback_providers</span><span class="p">))</span>
<span class="bp">self</span><span class="o">.</span><span class="n">set_providers</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_fallback_providers</span><span class="p">)</span>
<span class="c1"># Fallback only once.</span>
<span class="bp">self</span><span class="o">.</span><span class="n">disable_fallback</span><span class="p">()</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_sess</span><span class="o">.</span><span class="n">run</span><span class="p">(</span><span class="n">output_names</span><span class="p">,</span> <span class="n">input_feed</span><span class="p">,</span> <span class="n">run_options</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">raise</span></div>
<div class="viewcode-block" id="InferenceSession.end_profiling"><a class="viewcode-back" href="../../../api_summary.html#onnxruntime.InferenceSession.end_profiling">[docs]</a> <span class="k">def</span> <span class="nf">end_profiling</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd"> End profiling and return results in a file.</span>
<span class="sd"> The results are stored in a filename if the option</span>
<span class="sd"> :meth:`onnxruntime.SessionOptions.enable_profiling`.</span>
<span class="sd"> &quot;&quot;&quot;</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_sess</span><span class="o">.</span><span class="n">end_profiling</span><span class="p">()</span></div></div>
</pre></div>
</div>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<p class="logo"><a href="../../../index.html">
<img class="logo" src="../../../_static/ONNX_Runtime_icon.png" alt="Logo"/>
</a></p>
<h1 class="logo"><a href="../../../index.html">ONNX Runtime</a></h1>
<h3>Navigation</h3>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../../tutorial.html">Tutorial</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../api_summary.html">API Summary</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../auto_examples/index.html">Gallery of examples</a></li>
</ul>
<div class="relations">
<h3>Related Topics</h3>
<ul>
<li><a href="../../../index.html">Documentation overview</a><ul>
<li><a href="../../index.html">Module code</a><ul>
</ul></li>
</ul></li>
</ul>
</div>
<div id="searchbox" style="display: none" role="search">
<h3 id="searchlabel">Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="../../../search.html" method="get">
<input type="text" name="q" aria-labelledby="searchlabel" />
<input type="submit" value="Go" />
</form>
</div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="footer">
&copy;2018-2019, Microsoft.
|
Powered by <a href="http://sphinx-doc.org/">Sphinx 2.2.1</a>
&amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.12</a>
</div>
</body>
</html>
Reference in a new issue View git blame Copy permalink