Namespace Microsoft.ML.OnnxRuntime

Classes

DisposableNamedOnnxValue

This class serves as a container for model run output values including tensors, sequences of tensors, sequences and maps. It extends NamedOnnxValue, exposes the OnnxValueType and Tensor type The class must be disposed of. It disposes of _ortValueHolder that owns the underlying Ort output value and anything else that would need to be disposed by the instance of the class. Use factory method CreateFromOrtValue to obtain an instance of the class.

FixedBufferOnnxValue

Represents an OrtValue with its underlying buffer pinned

InferenceSession

Represents an Inference Session on an ONNX Model. This is a IDisposable class and it must be disposed of using either a explicit call to Dispose() method or a pattern of using() block. If this is a member of another class that class must also become IDisposable and it must dispose of InferfenceSession in its Dispose() method.

ModelMetadata

A class that queries and caches model metadata and exposes it as properties

NamedOnnxValue

NodeMetadata

Resembles type and shape information of session-graph nodes, used for communicating the shape/type of input/output nodes

OnnxRuntimeException

The Exception that is thrown for errors related ton OnnxRuntime

OrtAllocator

The class exposes native internal allocator for Onnxruntime. This allocator enables you to allocate memory from the internal memory pools including device allocations. Useful for binding.

OrtArenaCfg

This class encapsulates arena configuration information that will be used to define the behavior of an arena based allocator See docs/C_API.md for more details

OrtCUDAProviderOptions

Holds the options for configuring a CUDA Execution Provider instance

OrtEnv

This class initializes the process-global ONNX Runtime environment instance (OrtEnv)

OrtExternalAllocation

This class represents an arbitrary buffer of memory allocated and owned by the user. It can be either a CPU, GPU or other device memory that can be suitably represented by IntPtr. This is just a composite of the buffer related information. The memory is assumed to be pinned if necessary and usable immediately in the native code.

OrtIoBinding

This class enables binding of inputs and/or outputs to pre-allocated memory. This enables interesting scenarios. For example, if your input already resides in some pre-allocated memory like GPU, you can bind that piece of memory to an input name and shape and onnxruntime will use that as input. Other traditional inputs can also be bound that already exists as Tensors.

Note, that this arrangement is designed to minimize data copies and to that effect your memory allocations must match what is expected by the model, whether you run on CPU or GPU. Data copy will still be made, if your pre-allocated memory location does not match the one expected by the model. However, copies with OrtIoBindings are only done once, at the time of the binding, not at run time. This means, that if your input data required a copy, your further input modifications would not be seen by onnxruntime unless you rebind it, even if it is the same buffer. If you require the scenario where data is copied, OrtIOBinding may not be the best match for your use case.

The fact that data copy is not made during runtime also has performance implications.

OrtMemoryAllocation

This class represents memory allocation made by a specific onnxruntime allocator. Use OrtAllocator.Allocate() to obtain an instance of this class. It implements IDisposable and makes use of the original allocator used to allocate the memory. The lifespan of the allocator instance must eclipse the lifespan of the allocation. Or, if you prefer, all OrtMemoryAllocation instances must be disposed of before the corresponding allocator instances are disposed of.

OrtMemoryInfo

This class encapsulates and most of the time owns the underlying native OrtMemoryInfo instance. Instance returned from OrtAllocator will not own OrtMemoryInfo, the class must be disposed regardless.

Use this class to query and create OrtAllocator instances so you can pre-allocate memory for model inputs/outputs and use it for binding. Instances of the class can also used to created OrtValues bound to pre-allocated memory. In that case, the instance of OrtMemoryInfo contains the information about the allocator used to allocate the underlying memory.

OrtTensorRTProviderOptions

Holds the options for configuring a TensorRT Execution Provider instance

OrtValue

Represents a disposable OrtValue. This class exposes a native instance of OrtValue. The class implements IDisposable via SafeHandle and must be disposed.

PrePackedWeightsContainer

This class holds pre-packed weights of shared initializers to be shared across sessions using these initializers and thereby provide memory savings by sharing the same pre-packed versions of these shared initializers

ProviderOptionsValueHelper

This helper class contains methods to handle values of provider options

RunOptions

Sets various runtime options.

SessionOptions

Holds the options for creating an InferenceSession

SessionOptionsContainer

Helper to allow the creation/addition of session options based on pre-defined named entries.

Structs

OrtApi

OrtApiBase

Interfaces

IDisposableReadOnlyCollection<T>

Enums

CoreMLFlags

CoreML flags for use with SessionOptions

ExecutionMode

Controls whether you want to execute operators in the graph sequentially or in parallel. Usually when the model has many branches, setting this option to ExecutionMode.ORT_PARALLEL will give you better performance. See [ONNX_Runtime_Perf_Tuning.md] for more details.