Note
Click here to download the full example code
Profile the execution of a simple model¶
ONNX Runtime can profile the execution of the model. This example shows how to interpret the results.
import onnxruntime as rt
import numpy
from onnxruntime.datasets import get_example
Let’s load a very simple model and compute some prediction.
example1 = get_example("mul_1.pb")
sess = rt.InferenceSession(example1)
input_name = sess.get_inputs()[0].name
x = numpy.array([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]], dtype=numpy.float32)
res = sess.run(None, {input_name: x})
print(res)
Out:
[array([[ 1., 4.],
[ 9., 16.],
[25., 36.]], dtype=float32)]
We need to enable to profiling before running the predictions.
options = rt.SessionOptions()
options.enable_profiling = True
sess_profile = rt.InferenceSession(example1, options)
input_name = sess.get_inputs()[0].name
x = numpy.array([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]], dtype=numpy.float32)
sess.run(None, {input_name: x})
prof_file = sess_profile.end_profiling()
print(prof_file)
Out:
onnxruntime_profile__2019-01-10_16-09-08.json
The results are stored un a file in JSON format. Let’s see what it contains.
import json
with open(prof_file, "r") as f:
sess_time = json.load(f)
import pprint
pprint.pprint(sess_time)
Out:
[{'args': {},
'cat': 'Session',
'dur': 400,
'name': 'model_loading_uri',
'ph': 'X',
'pid': 16232,
'tid': 7452,
'ts': 30},
{'args': {},
'cat': 'Session',
'dur': 145,
'name': 'session_initialization',
'ph': 'X',
'pid': 16232,
'tid': 7452,
'ts': 452}]
Total running time of the script: ( 0 minutes 0.011 seconds)