API Summary¶

Device¶

The package is compiled for a specific device, GPU or CPU. The CPU implementation includes optimizations such as MKL (Math Kernel Libary). The following function indicates the chosen option:

onnxruntime.get_device() → str¶

Return the device used to compute the prediction (CPU, MKL, …)

Examples and datasets¶

The package contains a few models stored in ONNX format used in the documentation. These don’t need to be downloaded as they are installed with the package.

onnxruntime.datasets.get_example(name)[source]¶

Retrieves the absolute file name of an example.

Load and run a model¶

ONNX Runtime reads a model saved in ONNX format. The main class InferenceSession wraps these functionalities in a single place.

class onnxruntime.ModelMetadata¶

Pre-defined and custom metadata about the model. It is usually used to identify the model used to run the prediction and facilitate the comparison.

custom_metadata_map¶

additional metadata

description¶

description of the model

domain¶

ONNX domain

graph_name¶

graph name

producer_name¶

producer name

version¶

version of the model

class onnxruntime.InferenceSession(path_or_bytes, sess_options=None)[source]¶

This is the main class used to run a model.

end_profiling()[source]¶

End profiling and return results in a file.

The results are stored in a filename if the option onnxruntime.SessionOptions.enable_profiling().

get_inputs()[source]¶

Return the inputs metadata as a list of onnxruntime.NodeArg.

get_modelmeta()[source]¶

Return the metadata. See onnxruntime.ModelMetadata.

get_outputs()[source]¶

Return the outputs metadata as a list of onnxruntime.NodeArg.

run(output_names, input_feed, run_options=None)[source]¶

Compute the predictions.

Parameters:	output_names – name of the outputs input_feed – dictionary { input_name: input_value } run_options – See onnxruntime.RunOptions.

sess.run([output_name], {input_name: x})

class onnxruntime.NodeArg¶

Node argument definition, for both input and output, including arg name, arg type (contains both type and shape).

name¶

node name

shape¶

node shape (assuming the node holds a tensor)

type¶

node type

class onnxruntime.RunOptions¶

Configuration information for a single Run.

run_log_verbosity_level¶

Applies to a particular Run() invocation.

run_tag¶

To identify logs generated by a particular Run() invocation.

terminate¶

Set to True to terminate any currently executing calls that are using this RunOptions instance. The individual calls will exit gracefully and return an error status.

class onnxruntime.SessionOptions¶

Configuration information for a session.

enable_cpu_mem_arena¶

Enables the memory arena on CPU. Arena may pre-allocate memory for future usage. Set this option to false if you don’t want it. Default is True.

enable_mem_pattern¶

Enables the memory pattern optimization. The idea is if the input shapes are the same, we could trace the internal memory allocation and generate a memory pattern for future request. So next time we could just do one allocation with a big chunk for all the internal memory allocation. Default is true.

enable_profiling¶

Enable profiling for this session. Default is false.

enable_sequential_execution¶

Enables sequential execution, disables parallel execution. Default is true.

max_num_graph_transformation_steps¶

Runs optimization steps on the execution graph. Default is 5.

session_log_verbosity_level¶

Applies to session load, initialization, etc. Default is 0.

session_logid¶

Logger id to use for session output.

session_thread_pool_size¶

How many threads in the session thread pool. Default is 0 to let onnxruntime choose. This parameter is unused unless enable_sequential_execution is false.

Backend¶

In addition to the regular API which is optimized for performance and usability, ONNX Runtime also implements the ONNX backend API for verification of ONNX specification conformance. The following functions are supported:

onnxruntime.backend.is_compatible(*args, **kwargs)¶

Return whether the model is compatible with the backend.

Parameters:	model – unused device – None to use the default device or a string (ex: ‘CPU’)
Returns:	boolean

onnxruntime.backend.prepare(*args, **kwargs)¶

Load the model and creates a onnxruntime.InferenceSession ready to be used as a backend.

Parameters:	model – ModelProto (returned by onnx.load), string for a filename or bytes for a serialized model device – requested device for the computation, None means the default one which depends on the compilation settings kwargs – see onnxruntime.SessionOptions
Returns:	onnxruntime.InferenceSession

onnxruntime.backend.run(*args, **kwargs)¶

Compute the prediction.

Parameters:	model – onnxruntime.InferenceSession returned by function prepare inputs – inputs device – requested device for the computation, None means the default one which depends on the compilation settings kwargs – see onnxruntime.RunOptions
Returns:	predictions

onnxruntime.backend.supports_device(*args, **kwargs)¶

Check whether the backend is compiled with particular device support. In particular it’s used in the testing suite.