Show / Hide Table of Contents

# Class OrtIoBinding

This class enables binding of inputs and/or outputs to pre-allocated memory. This enables interesting scenarios. For example, if your input already resides in some pre-allocated memory like GPU, you can bind that piece of memory to an input name and shape and onnxruntime will use that as input. Other traditional inputs can also be bound that already exists as Tensors.

Note, that this arrangement is designed to minimize data copies and to that effect your memory allocations must match what is expected by the model, whether you run on CPU or GPU. Data copy will still be made, if your pre-allocated memory location does not match the one expected by the model. However, copies with OrtIoBindings are only done once, at the time of the binding, not at run time. This means, that if your input data required a copy, your further input modifications would not be seen by onnxruntime unless you rebind it, even if it is the same buffer. If you require the scenario where data is copied, OrtIOBinding may not be the best match for your use case.

The fact that data copy is not made during runtime also has performance implications.

##### Inheritance

System.Object
OrtIoBinding

###### Namespace: Microsoft.ML.OnnxRuntime

###### Assembly: cs.temp.dll.dll

##### Syntax

    public class OrtIoBinding : SafeHandle

### Properties

#### IsInvalid

Overrides SafeHandle.IsInvalid

##### Declaration

    public override bool IsInvalid { get; }

##### Property Value

Type | Description
--- | ---
System.Boolean | returns true if handle is equal to Zero

### Methods

#### BindInput(String, FixedBufferOnnxValue)

Bind the input with the given name as an OrtValue Tensor allocated in pinned managed memory. Instance of FixedBufferOnnxValue owns the memory and should be alive until the end of execution.

##### Declaration

    public void BindInput(string name, FixedBufferOnnxValue fixedValue)

##### Parameters

Type | Name | Description
--- | --- | ---
System.String | name | name of input
FixedBufferOnnxValue | fixedValue |

#### BindInput(String, OrtExternalAllocation)

Bind externally (not from OrtAllocator) allocated memory as input. The model will read the specified input from that memory possibly avoiding the need to copy between devices. The user code continues to own the chunk of externally allocated memory, and the allocation should be alive until the end of execution.

##### Declaration

    public void BindInput(string name, OrtExternalAllocation allocation)

##### Parameters

Type | Name | Description
--- | --- | ---
System.String | name | name
OrtExternalAllocation | allocation | non ort allocated memory

#### BindInput(String, TensorElementType, Int64[], OrtMemoryAllocation)

Bind a piece of pre-allocated native memory as a OrtValue Tensor with a given shape to an input with a given name. The model will read the specified input from that memory possibly avoiding the need to copy between devices. OrtMemoryAllocation continues to own the chunk of native memory, and the allocation should be alive until the end of execution.

##### Declaration

    public void BindInput(string name, TensorElementType elementType, long[] shape, OrtMemoryAllocation allocation)

##### Parameters

Type | Name | Description
--- | --- | ---
System.String | name | of the input
TensorElementType | elementType | Tensor element type
System.Int64[] | shape |
OrtMemoryAllocation | allocation | native memory allocation

#### BindOutput(String, FixedBufferOnnxValue)

Bind model output to a given instance of FixedBufferOnnxValue which owns the underlying pinned managed memory and should be alive for the time of execution.

##### Declaration

    public void BindOutput(string name, FixedBufferOnnxValue fixedValue)

##### Parameters

Type | Name | Description
--- | --- | ---
System.String | name | of the output
FixedBufferOnnxValue | fixedValue |

#### BindOutput(String, OrtExternalAllocation)

Bind externally (not from OrtAllocator) allocated memory as output. The model will read the specified input from that memory possibly avoiding the need to copy between devices. The user code continues to own the chunk of externally allocated memory, and the allocation should be alive until the end of execution.

##### Declaration

    public void BindOutput(string name, OrtExternalAllocation allocation)

##### Parameters

Type | Name | Description
--- | --- | ---
System.String | name | name
OrtExternalAllocation | allocation | non ort allocated memory

#### BindOutput(String, TensorElementType, Int64[], OrtMemoryAllocation)

Bind model output to an OrtValue as Tensor with a given type and shape. An instance of OrtMemoryAllocaiton owns the memory and should be alive for the time of execution.

##### Declaration

    public void BindOutput(string name, TensorElementType elementType, long[] shape, OrtMemoryAllocation allocation)

##### Parameters

Type | Name | Description
--- | --- | ---
System.String | name | of the output
TensorElementType | elementType | tensor element type
System.Int64[] | shape | tensor shape
OrtMemoryAllocation | allocation | allocated memory

#### BindOutputToDevice(String, OrtMemoryInfo)

This function will bind model output with the given name to a device specified by the memInfo.

##### Declaration

    public void BindOutputToDevice(string name, OrtMemoryInfo memInfo)

##### Parameters

Type | Name | Description
--- | --- | ---
System.String | name | output name
OrtMemoryInfo | memInfo | instance of memory info

#### ClearBoundInputs()

Clear all bound inputs and start anew

##### Declaration

    public void ClearBoundInputs()

#### ClearBoundOutputs()

Clear all bound outputs

##### Declaration

    public void ClearBoundOutputs()

#### GetOutputNames()

Returns an array of output names in the same order they were bound

##### Declaration

    public string[] GetOutputNames()

##### Returns

Type | Description
--- | ---
System.String[] | array of output names

#### GetOutputValues()

##### Declaration

    public IDisposableReadOnlyCollection<OrtValue> GetOutputValues()

##### Returns

Type | Description
--- | ---
IDisposableReadOnlyCollection<OrtValue> |

#### ReleaseHandle()

Overrides SafeHandle.ReleaseHandle() to properly dispose of the native instance of OrtIoBidning

##### Declaration

    protected override bool ReleaseHandle()

##### Returns

Type | Description
--- | ---
System.Boolean | always returns true

#### SynchronizeBoundInputs()

Blocks until device completes all preceding requested tasks. Useful for memory synchronization.

##### Declaration

    public void SynchronizeBoundInputs()

#### SynchronizeBoundOutputs()

Blocks until device completes all preceding requested tasks. Useful for memory synchronization.

##### Declaration

    public void SynchronizeBoundOutputs()