Show / Hide Table of Contents

# Class InferenceSession

Represents an Inference Session on an ONNX Model. This is a IDisposable class and it must be disposed of using either a explicit call to Dispose() method or a pattern of using() block. If this is a member of another class that class must also become IDisposable and it must dispose of InferfenceSession in its Dispose() method.

##### Inheritance

System.Object
InferenceSession

###### Namespace: Microsoft.ML.OnnxRuntime

###### Assembly: cs.temp.dll.dll

##### Syntax

    public class InferenceSession : IDisposable

### Constructors

#### InferenceSession(Byte[])

Constructs an InferenceSession from a model data in byte array

##### Declaration

    public InferenceSession(byte[] model)

##### Parameters

Type | Name | Description
--- | --- | ---
System.Byte[] | model |

#### InferenceSession(Byte[], PrePackedWeightsContainer)

Constructs an InferenceSession from a model data (in byte array) and it will use the provided pre-packed weights container to store and share pre-packed buffers of shared initializers across sessions if any.

##### Declaration

    public InferenceSession(byte[] model, PrePackedWeightsContainer prepackedWeightsContainer)

##### Parameters

Type | Name | Description
--- | --- | ---
System.Byte[] | model | Model as byte array
PrePackedWeightsContainer | prepackedWeightsContainer | Instance of PrepackedWeightsContainer. Lifetime of 'prepackedWeightsContainer' must be managed by the user and it must outlive any sessions reliant on it

#### InferenceSession(Byte[], SessionOptions)

Constructs an InferenceSession from a model data in byte array, with some additional session options

##### Declaration

    public InferenceSession(byte[] model, SessionOptions options)

##### Parameters

Type | Name | Description
--- | --- | ---
System.Byte[] | model |
SessionOptions | options |

#### InferenceSession(Byte[], SessionOptions, PrePackedWeightsContainer)

Constructs an InferenceSession from a model data (in byte array) with some additional session options and it will use the provided pre-packed weights container to store and share pre-packed buffers of shared initializers across sessions if any.

##### Declaration

    public InferenceSession(byte[] model, SessionOptions options, PrePackedWeightsContainer prepackedWeightsContainer)

##### Parameters

Type | Name | Description
--- | --- | ---
System.Byte[] | model | Model as byte array
SessionOptions | options | Session Options
PrePackedWeightsContainer | prepackedWeightsContainer | Instance of PrepackedWeightsContainer. Lifetime of 'prepackedWeightsContainer' must be managed by the user and it must outlive any sessions reliant on it

#### InferenceSession(String)

Constructs an InferenceSession from a model file

##### Declaration

    public InferenceSession(string modelPath)

##### Parameters

Type | Name | Description
--- | --- | ---
System.String | modelPath |

#### InferenceSession(String, PrePackedWeightsContainer)

Constructs an InferenceSession from a model file and it will use the provided pre-packed weights container to store and share pre-packed buffers of shared initializers across sessions if any.

##### Declaration

    public InferenceSession(string modelPath, PrePackedWeightsContainer prepackedWeightsContainer)

##### Parameters

Type | Name | Description
--- | --- | ---
System.String | modelPath | Model path
PrePackedWeightsContainer | prepackedWeightsContainer | Instance of PrepackedWeightsContainer. Lifetime of 'prepackedWeightsContainer' must be managed by the user and it must outlive any sessions reliant on it

#### InferenceSession(String, SessionOptions)

Constructs an InferenceSession from a model file, with some additional session options

##### Declaration

    public InferenceSession(string modelPath, SessionOptions options)

##### Parameters

Type | Name | Description
--- | --- | ---
System.String | modelPath |
SessionOptions | options |

#### InferenceSession(String, SessionOptions, PrePackedWeightsContainer)

Constructs an InferenceSession from a model file, with some additional session options and it will use the provided pre-packed weights container to store and share pre-packed buffers of shared initializers across sessions if any.

##### Declaration

    public InferenceSession(string modelPath, SessionOptions options, PrePackedWeightsContainer prepackedWeightsContainer)

##### Parameters

Type | Name | Description
--- | --- | ---
System.String | modelPath | Model path
SessionOptions | options | Session options
PrePackedWeightsContainer | prepackedWeightsContainer | Instance of PrepackedWeightsContainer. Lifetime of 'prepackedWeightsContainer' must be managed by the user and it must outlive any sessions reliant on it

### Properties

#### InputMetadata

Meta data regarding the input nodes, keyed by input names

##### Declaration

    public IReadOnlyDictionary<string, NodeMetadata> InputMetadata { get; }

##### Property Value

Type | Description
--- | ---
IReadOnlyDictionary<System.String, NodeMetadata> |

#### ModelMetadata

This property queries model metadata, constructs an instance of ModelMetadata and caches it

##### Declaration

    public ModelMetadata ModelMetadata { get; }

##### Property Value

Type | Description
--- | ---
ModelMetadata | Instance of ModelMetdata

#### OutputMetadata

Metadata regarding the output nodes, keyed by output names

##### Declaration

    public IReadOnlyDictionary<string, NodeMetadata> OutputMetadata { get; }

##### Property Value

Type | Description
--- | ---
IReadOnlyDictionary<System.String, NodeMetadata> |

#### OverridableInitializerMetadata

Metadata regarding the overridable initializers, keyed by node names

##### Declaration

    public IReadOnlyDictionary<string, NodeMetadata> OverridableInitializerMetadata { get; }

##### Property Value

Type | Description
--- | ---
IReadOnlyDictionary<System.String, NodeMetadata> |

#### ProfilingStartTimeNs

Return the nanoseconds of profiling's start time On some platforms, this timer may not be as precise as nanoseconds For instance, on Windows and MacOS, the precision will be ~100ns

##### Declaration

    public ulong ProfilingStartTimeNs { get; }

##### Property Value

Type | Description
--- | ---
System.UInt64 |

### Methods

#### CreateIoBinding()

Create OrtIoBinding instance to bind pre-allocated buffers to input/output

##### Declaration

    public OrtIoBinding CreateIoBinding()

##### Returns

Type | Description
--- | ---
OrtIoBinding | A new instance of OrtIoBinding

#### Dispose()

IDisposable implementation

##### Declaration

    public void Dispose()

#### Dispose(Boolean)

IDisposable implementation

##### Declaration

    protected virtual void Dispose(bool disposing)

##### Parameters

Type | Name | Description
--- | --- | ---
System.Boolean | disposing | true if invoked from Dispose() method

#### EndProfiling()

Ends profiling for the session.

##### Declaration

    public string EndProfiling()

##### Returns

Type | Description
--- | ---
System.String | Returns the profile file name.

#### Finalize()

Finalizer. to cleanup session in case it runs and the user forgets to Dispose() of the session

##### Declaration

    protected void Finalize()

#### Run(IReadOnlyCollection<NamedOnnxValue>)

Runs the loaded model for the given inputs, and fetches all the outputs.

##### Declaration

    public IDisposableReadOnlyCollection<DisposableNamedOnnxValue> Run(IReadOnlyCollection<NamedOnnxValue> inputs)

##### Parameters

Type | Name | Description
--- | --- | ---
IReadOnlyCollection<NamedOnnxValue> | inputs | specify a collection of NamedOnnxValue that indicates the input values.

##### Returns

Type | Description
--- | ---
IDisposableReadOnlyCollection<DisposableNamedOnnxValue> | Output Tensors in a Collection of NamedOnnxValue. User must dispose the output.

#### Run(IReadOnlyCollection<NamedOnnxValue>, IReadOnlyCollection<NamedOnnxValue>)

Runs the loaded model for the given inputs and outputs.

Outputs need to be created with correct type and dimension to receive the fetched data.

##### Declaration

    public void Run(IReadOnlyCollection<NamedOnnxValue> inputs, IReadOnlyCollection<NamedOnnxValue> outputs)

##### Parameters

Type | Name | Description
--- | --- | ---
IReadOnlyCollection<NamedOnnxValue> | inputs | Specify a collection of NamedOnnxValue that indicates the input values.
IReadOnlyCollection<NamedOnnxValue> | outputs | Specify a collection of NamedOnnxValue that indicates the output values.

#### Run(IReadOnlyCollection<NamedOnnxValue>, IReadOnlyCollection<NamedOnnxValue>, RunOptions)

Runs the loaded model for the given inputs and outputs. Uses the given RunOptions for this run.

Outputs need to be created with correct type and dimension to receive the fetched data.

##### Declaration

    public void Run(IReadOnlyCollection<NamedOnnxValue> inputs, IReadOnlyCollection<NamedOnnxValue> outputs, RunOptions options)

##### Parameters

Type | Name | Description
--- | --- | ---
IReadOnlyCollection<NamedOnnxValue> | inputs | Specify a collection of NamedOnnxValue that indicates the input values.
IReadOnlyCollection<NamedOnnxValue> | outputs | Specify a collection of NamedOnnxValue that indicates the output values.
RunOptions | options |

#### Run(IReadOnlyCollection<NamedOnnxValue>, IReadOnlyCollection<String>)

Runs the loaded model for the given inputs, and fetches the outputs specified in `outputNames`.

##### Declaration

    public IDisposableReadOnlyCollection<DisposableNamedOnnxValue> Run(IReadOnlyCollection<NamedOnnxValue> inputs, IReadOnlyCollection<string> outputNames)

##### Parameters

Type | Name | Description
--- | --- | ---
IReadOnlyCollection<NamedOnnxValue> | inputs | Specify a collection of NamedOnnxValue that indicates the input values.
IReadOnlyCollection<System.String> | outputNames | Specify a collection of string that indicates the output names to fetch.

##### Returns

Type | Description
--- | ---
IDisposableReadOnlyCollection<DisposableNamedOnnxValue> | Output Tensors in a Collection of NamedOnnxValue. User must dispose the output.

#### Run(IReadOnlyCollection<NamedOnnxValue>, IReadOnlyCollection<String>, IReadOnlyCollection<FixedBufferOnnxValue>)

Runs the loaded model for the given inputs and outputs.

Outputs need to be created with correct type and dimension to receive the fetched data.

##### Declaration

    public void Run(IReadOnlyCollection<NamedOnnxValue> inputs, IReadOnlyCollection<string> outputNames, IReadOnlyCollection<FixedBufferOnnxValue> outputValues)

##### Parameters

Type | Name | Description
--- | --- | ---
IReadOnlyCollection<NamedOnnxValue> | inputs | Specify a collection of NamedOnnxValue that indicates the input values.
IReadOnlyCollection<System.String> | outputNames | Specify a collection of string that indicates the output names. Should match `outputValues`.
IReadOnlyCollection<FixedBufferOnnxValue> | outputValues | Specify a collection of FixedBufferOnnxValue that indicates the output values.

#### Run(IReadOnlyCollection<NamedOnnxValue>, IReadOnlyCollection<String>, IReadOnlyCollection<FixedBufferOnnxValue>, RunOptions)

Runs the loaded model for the given inputs and outputs. Uses the given RunOptions for this run.

Outputs need to be created with correct type and dimension to receive the fetched data.

##### Declaration

    public void Run(IReadOnlyCollection<NamedOnnxValue> inputs, IReadOnlyCollection<string> outputNames, IReadOnlyCollection<FixedBufferOnnxValue> outputValues, RunOptions options)

##### Parameters

Type | Name | Description
--- | --- | ---
IReadOnlyCollection<NamedOnnxValue> | inputs | Specify a collection of NamedOnnxValue that indicates the input values.
IReadOnlyCollection<System.String> | outputNames | Specify a collection of string that indicates the output names. Should match `outputValues`.
IReadOnlyCollection<FixedBufferOnnxValue> | outputValues | Specify a collection of FixedBufferOnnxValue that indicates the output values.
RunOptions | options |

#### Run(IReadOnlyCollection<NamedOnnxValue>, IReadOnlyCollection<String>, RunOptions)

Runs the loaded model for the given inputs, and fetches the specified outputs in `outputNames`. Uses the given RunOptions for this run.

##### Declaration

    public IDisposableReadOnlyCollection<DisposableNamedOnnxValue> Run(IReadOnlyCollection<NamedOnnxValue> inputs, IReadOnlyCollection<string> outputNames, RunOptions options)

##### Parameters

Type | Name | Description
--- | --- | ---
IReadOnlyCollection<NamedOnnxValue> | inputs | Specify a collection of NamedOnnxValue that indicates the input values.
IReadOnlyCollection<System.String> | outputNames | Specify a collection of string that indicates the output names to fetch.
RunOptions | options |

##### Returns

Type | Description
--- | ---
IDisposableReadOnlyCollection<DisposableNamedOnnxValue> | Output Tensors in a Collection of NamedOnnxValue. User must dispose the output.

#### Run(IReadOnlyCollection<String>, IReadOnlyCollection<FixedBufferOnnxValue>)

Runs the loaded model for the given inputs, and fetches all the outputs.

##### Declaration

    public IDisposableReadOnlyCollection<DisposableNamedOnnxValue> Run(IReadOnlyCollection<string> inputNames, IReadOnlyCollection<FixedBufferOnnxValue> inputValues)

##### Parameters

Type | Name | Description
--- | --- | ---
IReadOnlyCollection<System.String> | inputNames | Specify a collection of string that indicates the input names. Should match `inputValues`.
IReadOnlyCollection<FixedBufferOnnxValue> | inputValues | Specify a collection of FixedBufferOnnxValue that indicates the input values.

##### Returns

Type | Description
--- | ---
IDisposableReadOnlyCollection<DisposableNamedOnnxValue> | Output Tensors in a Collection of NamedOnnxValue. User must dispose the output.

#### Run(IReadOnlyCollection<String>, IReadOnlyCollection<FixedBufferOnnxValue>, IReadOnlyCollection<NamedOnnxValue>)

Runs the loaded model for the given inputs and outputs.

Outputs need to be created with correct type and dimension to receive the fetched data.

##### Declaration

    public void Run(IReadOnlyCollection<string> inputNames, IReadOnlyCollection<FixedBufferOnnxValue> inputValues, IReadOnlyCollection<NamedOnnxValue> outputs)

##### Parameters

Type | Name | Description
--- | --- | ---
IReadOnlyCollection<System.String> | inputNames | Specify a collection of string that indicates the input names. Should match `inputValues`.
IReadOnlyCollection<FixedBufferOnnxValue> | inputValues | Specify a collection of FixedBufferOnnxValue that indicates the input values.
IReadOnlyCollection<NamedOnnxValue> | outputs | Specify a collection of NamedOnnxValue that indicates the output values.

#### Run(IReadOnlyCollection<String>, IReadOnlyCollection<FixedBufferOnnxValue>, IReadOnlyCollection<NamedOnnxValue>, RunOptions)

Runs the loaded model for the given inputs and outputs. Uses the given RunOptions for this run.

Outputs need to be created with correct type and dimension to receive the fetched data.

##### Declaration

    public void Run(IReadOnlyCollection<string> inputNames, IReadOnlyCollection<FixedBufferOnnxValue> inputValues, IReadOnlyCollection<NamedOnnxValue> outputs, RunOptions options)

##### Parameters

Type | Name | Description
--- | --- | ---
IReadOnlyCollection<System.String> | inputNames | Specify a collection of string that indicates the input names. Should match `inputValues`.
IReadOnlyCollection<FixedBufferOnnxValue> | inputValues | Specify a collection of FixedBufferOnnxValue that indicates the input values.
IReadOnlyCollection<NamedOnnxValue> | outputs | Specify a collection of NamedOnnxValue that indicates the output values.
RunOptions | options |

#### Run(IReadOnlyCollection<String>, IReadOnlyCollection<FixedBufferOnnxValue>, IReadOnlyCollection<String>)

Runs the loaded model for the given inputs, and fetches the outputs specified in `outputNames`.

##### Declaration

    public IDisposableReadOnlyCollection<DisposableNamedOnnxValue> Run(IReadOnlyCollection<string> inputNames, IReadOnlyCollection<FixedBufferOnnxValue> inputValues, IReadOnlyCollection<string> outputNames)

##### Parameters

Type | Name | Description
--- | --- | ---
IReadOnlyCollection<System.String> | inputNames | Specify a collection of string that indicates the input names. Should match `inputValues`.
IReadOnlyCollection<FixedBufferOnnxValue> | inputValues | Specify a collection of FixedBufferOnnxValue that indicates the input values.
IReadOnlyCollection<System.String> | outputNames | Specify a collection of string that indicates the output names to fetch.

##### Returns

Type | Description
--- | ---
IDisposableReadOnlyCollection<DisposableNamedOnnxValue> | Output Tensors in a Collection of NamedOnnxValue. User must dispose the output.

#### Run(IReadOnlyCollection<String>, IReadOnlyCollection<FixedBufferOnnxValue>, IReadOnlyCollection<String>, IReadOnlyCollection<FixedBufferOnnxValue>)

Runs the loaded model for the given inputs and outputs.

Outputs need to be created with correct type and dimension to accept the fetched data.

##### Declaration

    public void Run(IReadOnlyCollection<string> inputNames, IReadOnlyCollection<FixedBufferOnnxValue> inputValues, IReadOnlyCollection<string> outputNames, IReadOnlyCollection<FixedBufferOnnxValue> outputValues)

##### Parameters

Type | Name | Description
--- | --- | ---
IReadOnlyCollection<System.String> | inputNames | Specify a collection of string that indicates the input names. Should match `inputValues`.
IReadOnlyCollection<FixedBufferOnnxValue> | inputValues | Specify a collection of FixedBufferOnnxValue that indicates the input values.
IReadOnlyCollection<System.String> | outputNames | Specify a collection of string that indicates the output names. Should match `outputValues`.
IReadOnlyCollection<FixedBufferOnnxValue> | outputValues | Specify a collection of FixedBufferOnnxValue that indicates the output values.

#### Run(IReadOnlyCollection<String>, IReadOnlyCollection<FixedBufferOnnxValue>, IReadOnlyCollection<String>, IReadOnlyCollection<FixedBufferOnnxValue>, RunOptions)

Runs the loaded model for the given inputs and outputs. Uses the given RunOptions for this run.

Outputs need to be created with correct type and dimension to accept the fetched data.

##### Declaration

    public void Run(IReadOnlyCollection<string> inputNames, IReadOnlyCollection<FixedBufferOnnxValue> inputValues, IReadOnlyCollection<string> outputNames, IReadOnlyCollection<FixedBufferOnnxValue> outputValues, RunOptions options)

##### Parameters

Type | Name | Description
--- | --- | ---
IReadOnlyCollection<System.String> | inputNames | Specify a collection of string that indicates the input names. Should match `inputValues`.
IReadOnlyCollection<FixedBufferOnnxValue> | inputValues | Specify a collection of FixedBufferOnnxValue that indicates the input values.
IReadOnlyCollection<System.String> | outputNames | Specify a collection of string that indicates the output names. Should match `outputValues`.
IReadOnlyCollection<FixedBufferOnnxValue> | outputValues | Specify a collection of FixedBufferOnnxValue that indicates the output values.
RunOptions | options |

#### Run(IReadOnlyCollection<String>, IReadOnlyCollection<FixedBufferOnnxValue>, IReadOnlyCollection<String>, RunOptions)

Runs the loaded model for the given inputs, and fetches the specified outputs in `outputNames`. Uses the given RunOptions for this run.

##### Declaration

    public IDisposableReadOnlyCollection<DisposableNamedOnnxValue> Run(IReadOnlyCollection<string> inputNames, IReadOnlyCollection<FixedBufferOnnxValue> inputValues, IReadOnlyCollection<string> outputNames, RunOptions options)

##### Parameters

Type | Name | Description
--- | --- | ---
IReadOnlyCollection<System.String> | inputNames | Specify a collection of string that indicates the input names. Should match `inputValues`.
IReadOnlyCollection<FixedBufferOnnxValue> | inputValues | Specify a collection of FixedBufferOnnxValue that indicates the input values.
IReadOnlyCollection<System.String> | outputNames | Specify a collection of string that indicates the output names to fetch.
RunOptions | options |

##### Returns

Type | Description
--- | ---
IDisposableReadOnlyCollection<DisposableNamedOnnxValue> | Output Tensors in a Collection of NamedOnnxValue. User must dispose the output.

#### RunWithBinding(RunOptions, OrtIoBinding)

This method runs inference on the OrtIoBinding instance The method does not return anything. This is a lightweight version of RunWithBindingAndNames(). When you bind pre-allocated buffers to the output values you may not want to fetch the outputs since you already have access to them so you can spare the expense of fetching them and pairing with names. You can still fetch the outputs by calling OrtIOBinding.GetOutputValues()

##### Declaration

    public void RunWithBinding(RunOptions runOptions, OrtIoBinding ioBinding)

##### Parameters

Type | Name | Description
--- | --- | ---
RunOptions | runOptions | runOptions
OrtIoBinding | ioBinding | ioBinding instance to use

#### RunWithBindingAndNames(RunOptions, OrtIoBinding, String[])

This method return a collection of DisposableNamedOnnxValue as in other interfaces Query names from OrtIoBinding object and pair then with the array of OrtValues returned from OrtIoBinding.GetOutputValues()

##### Declaration

    public IDisposableReadOnlyCollection<DisposableNamedOnnxValue> RunWithBindingAndNames(RunOptions runOptions, OrtIoBinding ioBinding, string[] names = null)

##### Parameters

Type | Name | Description
--- | --- | ---
RunOptions | runOptions | RunOptions
OrtIoBinding | ioBinding | OrtIoBinding instance with bindings
System.String[] | names | optional parameter. If you already know the names of the outputs you can save a native call to retrieve output names. They will be paired with the returned OrtValues and combined into DisposbleNamedOnnxValues. Otherwise, the method will retrieve output names from the OrtIoBinding instance. It is an error if you supply a different number of names than the returned outputs

##### Returns

Type | Description
--- | ---
IDisposableReadOnlyCollection<DisposableNamedOnnxValue> | A disposable collection of DisposableNamedOnnxValue that encapsulate output OrtValues