# OnnxRuntime: OrtCUDAProviderOptions Struct Reference

Loading...
Searching...
No Matches
OrtCUDAProviderOptions Struct Reference
Global

CUDA Provider Options. More...

`#include <onnxruntime_c_api.h>`

## Public Member Functions

| OrtCUDAProviderOptions ()

## Public Attributes

int | device_id
| CUDA device Id Defaults to 0.
OrtCudnnConvAlgoSearch | cudnn_conv_algo_search
| CUDA Convolution algorithm search configuration. See enum OrtCudnnConvAlgoSearch for more details. Defaults to OrtCudnnConvAlgoSearchExhaustive.
size_t | gpu_mem_limit
| CUDA memory limit (To use all possible memory pass in maximum size_t) Defaults to SIZE_MAX.
int | arena_extend_strategy
| Strategy used to grow the memory arena 0 = kNextPowerOfTwo 1 = kSameAsRequested Defaults to 0.
int | do_copy_in_default_stream
| Flag indicating if copying needs to take place on the same stream as the compute stream in the CUDA EP 0 = Use separate streams for copying and compute. 1 = Use the same stream for copying and compute. Defaults to 1. WARNING: Setting this to 0 may result in data races for some models. Please see issue #4829 for more details.
int | has_user_compute_stream
| Flag indicating if there is a user provided compute stream Defaults to 0.
void * | user_compute_stream
| User provided compute stream. If provided, please set `has_user_compute_stream` to 1.
OrtArenaCfg * | default_memory_arena_cfg
| CUDA memory arena configuration parameters.
int | tunable_op_enabled
| Enable TunableOp. Set it to 1 to enable TunableOp. Otherwise, it is disabled by default. This option can be superseded by environment variable ORT_CUDA_TUNABLE_OP_ENABLED.

## Detailed Description

CUDA Provider Options.

See also
  OrtApi::SessionOptionsAppendExecutionProvider_CUDA

## Constructor & Destructor Documentation

## OrtCUDAProviderOptions()

OrtCUDAProviderOptions::OrtCUDAProviderOptions | ( |  | ) |

inline

## Member Data Documentation

## arena_extend_strategy

int OrtCUDAProviderOptions::arena_extend_strategy

Strategy used to grow the memory arena 0 = kNextPowerOfTwo
1 = kSameAsRequested
Defaults to 0.

Note
  If a OrtArenaCfg has been applied, it will override this field

## cudnn_conv_algo_search

OrtCudnnConvAlgoSearch OrtCUDAProviderOptions::cudnn_conv_algo_search

CUDA Convolution algorithm search configuration. See enum OrtCudnnConvAlgoSearch for more details. Defaults to OrtCudnnConvAlgoSearchExhaustive.

## default_memory_arena_cfg

OrtArenaCfg* OrtCUDAProviderOptions::default_memory_arena_cfg

CUDA memory arena configuration parameters.

## device_id

int OrtCUDAProviderOptions::device_id

CUDA device Id Defaults to 0.

## do_copy_in_default_stream

int OrtCUDAProviderOptions::do_copy_in_default_stream

Flag indicating if copying needs to take place on the same stream as the compute stream in the CUDA EP 0 = Use separate streams for copying and compute. 1 = Use the same stream for copying and compute. Defaults to 1. WARNING: Setting this to 0 may result in data races for some models. Please see issue #4829 for more details.

## gpu_mem_limit

size_t OrtCUDAProviderOptions::gpu_mem_limit

CUDA memory limit (To use all possible memory pass in maximum size_t) Defaults to SIZE_MAX.

Note
  If a OrtArenaCfg has been applied, it will override this field

## has_user_compute_stream

int OrtCUDAProviderOptions::has_user_compute_stream

Flag indicating if there is a user provided compute stream Defaults to 0.

## tunable_op_enabled

int OrtCUDAProviderOptions::tunable_op_enabled

Enable TunableOp. Set it to 1 to enable TunableOp. Otherwise, it is disabled by default. This option can be superseded by environment variable ORT_CUDA_TUNABLE_OP_ENABLED.

## user_compute_stream

void* OrtCUDAProviderOptions::user_compute_stream

User provided compute stream. If provided, please set `has_user_compute_stream` to 1.