Profile the execution of a simple model

ONNX Runtime can profile the execution of the model. This example shows how to interpret the results.

import onnxruntime as rt
import numpy
from onnxruntime.datasets import get_example

Let’s load a very simple model and compute some prediction.

example1 = get_example("mul_1.onnx")
sess = rt.InferenceSession(example1)
input_name = sess.get_inputs()[0].name

x = numpy.array([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]], dtype=numpy.float32)
res = sess.run(None, {input_name: x})
print(res)

Out:

[array([[ 1.,  4.],
       [ 9., 16.],
       [25., 36.]], dtype=float32)]

We need to enable to profiling before running the predictions.

options = rt.SessionOptions()
options.enable_profiling = True
sess_profile = rt.InferenceSession(example1, options)
input_name = sess.get_inputs()[0].name

x = numpy.array([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]], dtype=numpy.float32)

sess.run(None, {input_name: x})
prof_file = sess_profile.end_profiling()
print(prof_file)

Out:

onnxruntime_profile__2019-12-19_16-47-06.json

The results are stored un a file in JSON format. Let’s see what it contains.

import json
with open(prof_file, "r") as f:
    sess_time = json.load(f)
import pprint
pprint.pprint(sess_time)

Out:

[{'args': {},
  'cat': 'Session',
  'dur': 100,
  'name': 'model_loading_from_saved_proto',
  'ph': 'X',
  'pid': 27824,
  'tid': 13820,
  'ts': 10},
 {'args': {},
  'cat': 'Session',
  'dur': 200,
  'name': 'session_initialization',
  'ph': 'X',
  'pid': 27824,
  'tid': 13820,
  'ts': 123}]

Total running time of the script: ( 0 minutes 0.027 seconds)