Note
Click here to download the full example code
Profile the execution of a simple model¶
ONNX Runtime can profile the execution of the model. This example shows how to interpret the results.
import onnxruntime as rt
import numpy
from onnxruntime.datasets import get_example
Let’s load a very simple model and compute some prediction.
example1 = get_example("mul_1.onnx")
sess = rt.InferenceSession(example1)
input_name = sess.get_inputs()[0].name
x = numpy.array([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]], dtype=numpy.float32)
res = sess.run(None, {input_name: x})
print(res)
Out:
[array([[ 1., 4.],
[ 9., 16.],
[25., 36.]], dtype=float32)]
We need to enable to profiling before running the predictions.
options = rt.SessionOptions()
options.enable_profiling = True
sess_profile = rt.InferenceSession(example1, options)
input_name = sess.get_inputs()[0].name
x = numpy.array([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]], dtype=numpy.float32)
sess.run(None, {input_name: x})
prof_file = sess_profile.end_profiling()
print(prof_file)
Out:
onnxruntime_profile__2019-12-19_16-47-06.json
The results are stored un a file in JSON format. Let’s see what it contains.
import json
with open(prof_file, "r") as f:
sess_time = json.load(f)
import pprint
pprint.pprint(sess_time)
Out:
[{'args': {},
'cat': 'Session',
'dur': 100,
'name': 'model_loading_from_saved_proto',
'ph': 'X',
'pid': 27824,
'tid': 13820,
'ts': 10},
{'args': {},
'cat': 'Session',
'dur': 200,
'name': 'session_initialization',
'ph': 'X',
'pid': 27824,
'tid': 13820,
'ts': 123}]
Total running time of the script: ( 0 minutes 0.027 seconds)