Class SessionOptions
Holds the options for creating an InferenceSession
Inheritance
Namespace: Microsoft.ML.OnnxRuntime
Assembly: cs.temp.dll.dll
Syntax
public class SessionOptions : SafeHandleConstructors
SessionOptions()
Constructs an empty SessionOptions
Declaration
public SessionOptions()Properties
EnableCpuMemArena
Enables Arena allocator for the CPU memory allocations. Default is true.
Declaration
public bool EnableCpuMemArena { get; set; }Property Value
| Type | Description |
|---|---|
| System.Boolean | returns _enableCpuMemArena flag value |
EnableMemoryPattern
Enables the use of the memory allocation patterns in the first Run() call for subsequent runs. Default = true.
Declaration
public bool EnableMemoryPattern { get; set; }Property Value
| Type | Description |
|---|---|
| System.Boolean | returns enableMemoryPattern flag value |
EnableProfiling
Enables profiling of InferenceSession.Run() calls. Default is false
Declaration
public bool EnableProfiling { get; set; }Property Value
| Type | Description |
|---|---|
| System.Boolean | returns _enableProfiling flag value |
ExecutionMode
Sets the execution mode for the session. Default is set to ORT_SEQUENTIAL. See [ONNX_Runtime_Perf_Tuning.md] for more details.
Declaration
public ExecutionMode ExecutionMode { get; set; }Property Value
| Type | Description |
|---|---|
| ExecutionMode | returns _executionMode value |
GraphOptimizationLevel
Sets the graph optimization level for the session. Default is set to ORT_ENABLE_ALL.
Declaration
public GraphOptimizationLevel GraphOptimizationLevel { get; set; }Property Value
| Type | Description |
|---|---|
| GraphOptimizationLevel | returns _graphOptimizationLevel value |
InterOpNumThreads
Declaration
public int InterOpNumThreads { get; set; }Property Value
| Type | Description |
|---|---|
| System.Int32 |
IntraOpNumThreads
Declaration
public int IntraOpNumThreads { get; set; }Property Value
| Type | Description |
|---|---|
| System.Int32 |
IsInvalid
Overrides SafeHandle.IsInvalid
Declaration
public override bool IsInvalid { get; }Property Value
| Type | Description |
|---|---|
| System.Boolean | returns true if handle is equal to Zero |
LogId
Log Id to be used for the session. Default is empty string.
Declaration
public string LogId { get; set; }Property Value
| Type | Description |
|---|---|
| System.String | returns _logId value |
LogSeverityLevel
Log Severity Level for the session logs. Default = ORT_LOGGING_LEVEL_WARNING
Declaration
public OrtLoggingLevel LogSeverityLevel { get; set; }Property Value
| Type | Description |
|---|---|
| OrtLoggingLevel | returns _logSeverityLevel value |
LogVerbosityLevel
Log Verbosity Level for the session logs. Default = 0. Valid values are >=0. This takes into effect only when the LogSeverityLevel is set to ORT_LOGGING_LEVEL_VERBOSE.
Declaration
public int LogVerbosityLevel { get; set; }Property Value
| Type | Description |
|---|---|
| System.Int32 | returns _logVerbosityLevel value |
OptimizedModelFilePath
Set filepath to save optimized model after graph level transformations. Default is empty, which implies saving is disabled.
Declaration
public string OptimizedModelFilePath { get; set; }Property Value
| Type | Description |
|---|---|
| System.String | returns _optimizedModelFilePath flag value |
ProfileOutputPathPrefix
Path prefix to use for output of profiling data
Declaration
public string ProfileOutputPathPrefix { get; set; }Property Value
| Type | Description |
|---|---|
| System.String |
Methods
AddFreeDimensionOverride(String, Int64)
Override symbolic dimensions (by specific denotation strings) with actual values if known at session initialization time to enable optimizations that can take advantage of fixed values (such as memory planning, etc)
Declaration
public void AddFreeDimensionOverride(string dimDenotation, long dimValue)Parameters
| Type | Name | Description |
|---|---|---|
| System.String | dimDenotation | denotation name |
| System.Int64 | dimValue | denotation value |
AddFreeDimensionOverrideByName(String, Int64)
Override symbolic dimensions (by specific name strings) with actual values if known at session initialization time to enable optimizations that can take advantage of fixed values (such as memory planning, etc)
Declaration
public void AddFreeDimensionOverrideByName(string dimName, long dimValue)Parameters
| Type | Name | Description |
|---|---|---|
| System.String | dimName | dimension name |
| System.Int64 | dimValue | dimension value |
AddInitializer(String, OrtValue)
Add a pre-allocated initializer to a session. If a model contains an initializer with a name that is same as the name passed to this API call, ORT will use this initializer instance instead of deserializing one from the model file. This is useful when you want to share the same initializer across sessions.
Declaration
public void AddInitializer(string name, OrtValue ortValue)Parameters
| Type | Name | Description |
|---|---|---|
| System.String | name | name of the initializer |
| OrtValue | ortValue | OrtValue containing the initializer. Lifetime of 'val' and the underlying initializer buffer must be managed by the user (created using the CreateTensorWithDataAsOrtValue API) and it must outlive the session object |
AddSessionConfigEntry(String, String)
Set a single session configuration entry as a pair of strings If a configuration with same key exists, this will overwrite the configuration with the given configValue
Declaration
public void AddSessionConfigEntry(string configKey, string configValue)Parameters
| Type | Name | Description |
|---|---|---|
| System.String | configKey | config key name |
| System.String | configValue | config key value |
AppendExecutionProvider(String, Dictionary<String, String>)
Append SNPE or XNNPACK execution provider
Declaration
public void AppendExecutionProvider(string providerName, Dictionary<string, string> providerOptions = null)Parameters
| Type | Name | Description |
|---|---|---|
| System.String | providerName | Execution provider to add. 'SNPE' or 'XNNPACK' are currently supported. |
| Dictionary<System.String, System.String> | providerOptions | Optional key/value pairs to specify execution provider options. |
AppendExecutionProvider_CoreML(CoreMLFlags)
Use only if you have the onnxruntime package specific to this Execution Provider.
Declaration
public void AppendExecutionProvider_CoreML(CoreMLFlags coremlFlags = CoreMLFlags.COREML_FLAG_USE_NONE)Parameters
| Type | Name | Description |
|---|---|---|
| CoreMLFlags | coremlFlags | CoreML specific flags |
AppendExecutionProvider_CPU(Int32)
Appends CPU EP to a list of available execution providers for the session.
Declaration
public void AppendExecutionProvider_CPU(int useArena = 1)Parameters
| Type | Name | Description |
|---|---|---|
| System.Int32 | useArena | 1 - use arena, 0 - do not use arena |
AppendExecutionProvider_CUDA(OrtCUDAProviderOptions)
Append a CUDA EP instance (based on specified configuration) to the SessionOptions instance. Use only if you have the onnxruntime package specific to this Execution Provider.
Declaration
public void AppendExecutionProvider_CUDA(OrtCUDAProviderOptions cudaProviderOptions)Parameters
| Type | Name | Description |
|---|---|---|
| OrtCUDAProviderOptions | cudaProviderOptions | CUDA EP provider options |
AppendExecutionProvider_CUDA(Int32)
Use only if you have the onnxruntime package specific to this Execution Provider.
Declaration
public void AppendExecutionProvider_CUDA(int deviceId = 0)Parameters
| Type | Name | Description |
|---|---|---|
| System.Int32 | deviceId | integer device ID |
AppendExecutionProvider_DML(Int32)
Use only if you have the onnxruntime package specific to this Execution Provider.
Declaration
public void AppendExecutionProvider_DML(int deviceId = 0)Parameters
| Type | Name | Description |
|---|---|---|
| System.Int32 | deviceId | device identification |
AppendExecutionProvider_Dnnl(Int32)
Use only if you have the onnxruntime package specific to this Execution Provider.
Declaration
public void AppendExecutionProvider_Dnnl(int useArena = 1)Parameters
| Type | Name | Description |
|---|---|---|
| System.Int32 | useArena | 1 - use allocation arena, 0 - otherwise |
AppendExecutionProvider_MIGraphX(Int32)
Use only if you have the onnxruntime package specific to this Execution Provider.
Declaration
public void AppendExecutionProvider_MIGraphX(int deviceId = 0)Parameters
| Type | Name | Description |
|---|---|---|
| System.Int32 | deviceId | device identification |
AppendExecutionProvider_Nnapi(NnapiFlags)
Use only if you have the onnxruntime package specific to this Execution Provider.
Declaration
public void AppendExecutionProvider_Nnapi(NnapiFlags nnapiFlags = NnapiFlags.NNAPI_FLAG_USE_NONE)Parameters
| Type | Name | Description |
|---|---|---|
| NnapiFlags | nnapiFlags | NNAPI specific flag mask |
AppendExecutionProvider_OpenVINO(String)
Use only if you have the onnxruntime package specific to this Execution Provider.
Declaration
public void AppendExecutionProvider_OpenVINO(string deviceId = "")Parameters
| Type | Name | Description |
|---|---|---|
| System.String | deviceId | device identification, default empty string |
AppendExecutionProvider_ROCM(Int32)
Use only if you have the onnxruntime package specific to this Execution Provider.
Declaration
public void AppendExecutionProvider_ROCM(int deviceId = 0)Parameters
| Type | Name | Description |
|---|---|---|
| System.Int32 | deviceId | Device Id |
AppendExecutionProvider_Tensorrt(OrtTensorRTProviderOptions)
Append a TensorRT EP instance (based on specified configuration) to the SessionOptions instance. Use only if you have the onnxruntime package specific to this Execution Provider.
Declaration
public void AppendExecutionProvider_Tensorrt(OrtTensorRTProviderOptions trtProviderOptions)Parameters
| Type | Name | Description |
|---|---|---|
| OrtTensorRTProviderOptions | trtProviderOptions | TensorRT EP provider options |
AppendExecutionProvider_Tensorrt(Int32)
Use only if you have the onnxruntime package specific to this Execution Provider.
Declaration
public void AppendExecutionProvider_Tensorrt(int deviceId = 0)Parameters
| Type | Name | Description |
|---|---|---|
| System.Int32 | deviceId | device identification |
AppendExecutionProvider_Tvm(String)
Use only if you have the onnxruntime package specific to this Execution Provider.
Declaration
public void AppendExecutionProvider_Tvm(string settings = "")Parameters
| Type | Name | Description |
|---|---|---|
| System.String | settings | string with TVM specific settings |
MakeSessionOptionWithCudaProvider(OrtCUDAProviderOptions)
A helper method to construct a SessionOptions object for CUDA execution provider. Use only if CUDA is installed and you have the onnxruntime package specific to this Execution Provider.
Declaration
public static SessionOptions MakeSessionOptionWithCudaProvider(OrtCUDAProviderOptions cudaProviderOptions)Parameters
| Type | Name | Description |
|---|---|---|
| OrtCUDAProviderOptions | cudaProviderOptions | CUDA EP provider options |
Returns
| Type | Description |
|---|---|
| SessionOptions | A SessionsOptions() object configured for execution on provider options |
MakeSessionOptionWithCudaProvider(Int32)
A helper method to construct a SessionOptions object for CUDA execution. Use only if CUDA is installed and you have the onnxruntime package specific to this Execution Provider.
Declaration
public static SessionOptions MakeSessionOptionWithCudaProvider(int deviceId = 0)Parameters
| Type | Name | Description |
|---|---|---|
| System.Int32 | deviceId |
Returns
| Type | Description |
|---|---|
| SessionOptions | A SessionsOptions() object configured for execution on deviceId |
MakeSessionOptionWithRocmProvider(Int32)
A helper method to construct a SessionOptions object for ROCM execution. Use only if ROCM is installed and you have the onnxruntime package specific to this Execution Provider.
Declaration
public static SessionOptions MakeSessionOptionWithRocmProvider(int deviceId = 0)Parameters
| Type | Name | Description |
|---|---|---|
| System.Int32 | deviceId | Device Id |
Returns
| Type | Description |
|---|---|
| SessionOptions | A SessionsOptions() object configured for execution on deviceId |
MakeSessionOptionWithTensorrtProvider(OrtTensorRTProviderOptions)
A helper method to construct a SessionOptions object for TensorRT execution provider. Use only if CUDA/TensorRT are installed and you have the onnxruntime package specific to this Execution Provider.
Declaration
public static SessionOptions MakeSessionOptionWithTensorrtProvider(OrtTensorRTProviderOptions trtProviderOptions)Parameters
| Type | Name | Description |
|---|---|---|
| OrtTensorRTProviderOptions | trtProviderOptions | TensorRT EP provider options |
Returns
| Type | Description |
|---|---|
| SessionOptions | A SessionsOptions() object configured for execution on provider options |
MakeSessionOptionWithTensorrtProvider(Int32)
A helper method to construct a SessionOptions object for TensorRT execution. Use only if CUDA/TensorRT are installed and you have the onnxruntime package specific to this Execution Provider.
Declaration
public static SessionOptions MakeSessionOptionWithTensorrtProvider(int deviceId = 0)Parameters
| Type | Name | Description |
|---|---|---|
| System.Int32 | deviceId |
Returns
| Type | Description |
|---|---|
| SessionOptions | A SessionsOptions() object configured for execution on deviceId |
MakeSessionOptionWithTvmProvider(String)
A helper method to construct a SessionOptions object for TVM execution. Use only if you have the onnxruntime package specific to this Execution Provider.
Declaration
public static SessionOptions MakeSessionOptionWithTvmProvider(String settings = null)Parameters
| Type | Name | Description |
|---|---|---|
| String | settings | settings string, comprises of comma separated key:value pairs. default is empty |
Returns
| Type | Description |
|---|---|
| SessionOptions | A SessionsOptions() object configured for execution with TVM |
RegisterCustomOpLibrary(String)
(Deprecated) Loads a DLL named 'libraryPath' and looks for this entry point: OrtStatus* RegisterCustomOps(OrtSessionOptions* options, const OrtApiBase* api); It then passes in the provided session options to this function along with the api base. Deprecated in favor of RegisterCustomOpLibraryV2() because it provides users with the library handle to release when all sessions relying on it are destroyed
Declaration
public void RegisterCustomOpLibrary(string libraryPath)Parameters
| Type | Name | Description |
|---|---|---|
| System.String | libraryPath | path to the custom op library |
RegisterCustomOpLibraryV2(String, out IntPtr)
Loads a DLL named 'libraryPath' and looks for this entry point: OrtStatus* RegisterCustomOps(OrtSessionOptions* options, const OrtApiBase* api); It then passes in the provided session options to this function along with the api base. The handle to the loaded library is returned in 'libraryHandle'. It can be unloaded by the caller after all sessions using the passed in session options are destroyed, or if an error occurs and it is non null. Hint: .NET Core 3.1 has a 'NativeLibrary' class that can be used to free the library handle
Declaration
public void RegisterCustomOpLibraryV2(string libraryPath, out IntPtr libraryHandle)Parameters
| Type | Name | Description |
|---|---|---|
| System.String | libraryPath | Custom op library path |
| IntPtr | libraryHandle | out parameter, library handle |
ReleaseHandle()
Overrides SafeHandle.ReleaseHandle() to properly dispose of the native instance of SessionOptions
Declaration
protected override bool ReleaseHandle()Returns
| Type | Description |
|---|---|
| System.Boolean | always returns true |