ONNX Runtime for Keras¶

The following demonstrates how to compute the predictions of a pretrained deep learning model obtained from keras with onnxruntime. The conversion requires keras, tensorflow, keras-onnx, onnxmltools but then only onnxruntime is required to compute the predictions.

import os
if not os.path.exists('dense121.onnx'):
    from keras.applications.densenet import DenseNet121
    model = DenseNet121(include_top=True, weights='imagenet')

    from keras2onnx import convert_keras
    onx = convert_keras(model, 'dense121.onnx')
    with open("dense121.onnx", "wb") as f:
        f.write(onx.SerializeToString())

Let’s load an image (source: wikipedia).

from keras.preprocessing.image import array_to_img, img_to_array, load_img
img = load_img('Sannosawa1.jpg')
ximg = img_to_array(img)

import matplotlib.pyplot as plt
plt.imshow(ximg / 255)
plt.axis('off')

../_images/sphx_glr_plot_dl_keras_001.png

Let’s load the model with onnxruntime.

import onnxruntime as rt
from onnxruntime.capi.onnxruntime_pybind11_state import InvalidGraph

try:
    sess = rt.InferenceSession('dense121.onnx')
    ok = True
except (InvalidGraph, TypeError, RuntimeError) as e:
    # Probably a mismatch between onnxruntime and onnx version.
    print(e)
    ok = False

if ok:
    print("The model expects input shape:", sess.get_inputs()[0].shape)
    print("image shape:", ximg.shape)

Out:

The model expects input shape: ['N', 224, 224, 3]
image shape: (960, 1280, 3)

Let’s resize the image.

if ok:
    from skimage.transform import resize
    import numpy

    ximg224 = resize(ximg / 255, (224, 224, 3), anti_aliasing=True)
    ximg = ximg224[numpy.newaxis, :, :, :]
    ximg = ximg.astype(numpy.float32)

    print("new shape:", ximg.shape)

Out:

new shape: (1, 224, 224, 3)

Let’s compute the output.

if ok:
    input_name = sess.get_inputs()[0].name
    res = sess.run(None, {input_name: ximg})
    prob = res[0]
    print(prob.ravel()[:10])  # Too big to be displayed.

Out:

[5.3862054e-06 6.1927537e-08 2.0104133e-07 1.3239808e-06 1.0415122e-06
 6.7792627e-08 3.1348068e-07 3.2683040e-08 1.0914272e-07 1.1627122e-06]

Let’s get more comprehensive results.

if ok:
    from keras.applications.densenet import decode_predictions
    decoded = decode_predictions(prob)

    import pandas
    df = pandas.DataFrame(decoded[0], columns=["class_id", "name", "P"])
    print(df)

Out:

class_id        name         P
n09468604      valley  0.785549
n09193705         alp  0.187054
n09399592  promontory  0.010602
n09246464       cliff  0.005537
n02417914        ibex  0.001997

Total running time of the script: ( 0 minutes 56.895 seconds)

Gallery generated by Sphinx-Gallery