diff --git a/onnxruntime/python/tools/quantization/notebooks/imagenet_v2/mobilenet.ipynb b/onnxruntime/python/tools/quantization/notebooks/imagenet_v2/mobilenet.ipynb index e1bb201ef2..c3cacf63cf 100644 --- a/onnxruntime/python/tools/quantization/notebooks/imagenet_v2/mobilenet.ipynb +++ b/onnxruntime/python/tools/quantization/notebooks/imagenet_v2/mobilenet.ipynb @@ -19,7 +19,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "In this tutorial, we will load a mobilenet v2 model pretrained with [PyTorch](https://pytorch.org/), export the model to ONNX, and then quantize and run with ONNXRuntime." + "In this tutorial, we will load a mobilenet v2 model pretrained with [PyTorch](https://pytorch.org/), export the model to ONNX, quantize then run with ONNXRuntime, and convert the ONNX models to ORT format for ONNXRuntime Mobile." ] }, { @@ -30,7 +30,7 @@ "\n", "If you have Jupyter Notebook, you can run this notebook directly with it. You may need to install or upgrade [PyTorch](https://pytorch.org/), [OnnxRuntime](https://microsoft.github.io/onnxruntime/), and other required packages.\n", "\n", - "Otherwise, you can setup a new environment. First, install [AnaConda](https://www.anaconda.com/distribution/). Then open an AnaConda prompt window and run the following commands:\n", + "Otherwise, you can setup a new environment. First, install [Anaconda](https://www.anaconda.com/distribution/). Then open an AnaConda prompt window and run the following commands:\n", "\n", "```console\n", "conda create -n cpu_env python=3.8\n", @@ -46,7 +46,7 @@ "metadata": {}, "source": [ "### 0.1 Install packages\n", - "Let's install nessasary packages to start the tutorial. We will install PyTorch 1.8, OnnxRuntime 1.7, latest ONNX and pillow." + "Let's install the necessary packages to start the tutorial. We will install PyTorch 1.8, OnnxRuntime 1.8, latest ONNX and pillow." ] }, { @@ -57,10 +57,10 @@ }, "outputs": [], "source": [ - "# Install or upgrade PyTorch 1.8.0 and OnnxRuntime 1.7 for CPU-only.\n", + "# Install or upgrade PyTorch 1.8.0 and OnnxRuntime 1.8 for CPU-only.\n", "import sys\n", - "!{sys.executable} -m pip install --upgrade torch==1.8.0+cpu torchvision==0.9.0+cpu torchaudio===0.8.0 -f https://download.pytorch.org/whl/torch_stable.html\n", - "!{sys.executable} -m pip install --upgrade onnxruntime==1.7.0\n", + "!{sys.executable} -m pip install --upgrade torch==1.8.0 torchvision==0.9.0 torchaudio===0.8.0 -f https://download.pytorch.org/whl/torch_stable.html\n", + "!{sys.executable} -m pip install --upgrade onnxruntime==1.8.0\n", "!{sys.executable} -m pip install --upgrade onnx\n", "!{sys.executable} -m pip install --upgrade pillow" ] @@ -120,14 +120,12 @@ "# Export the model\n", "torch.onnx.export(mobilenet_v2, # model being run\n", " x, # model input (or a tuple for multiple inputs)\n", - " \"mobilenet_v2_float.onnx\", # where to save the model (can be a file or file-like object)\n", + " \"mobilenet_v2_float.onnx\", # where to save the model (can be a file or file-like object)\n", " export_params=True, # store the trained parameter weights inside the model file\n", " opset_version=12, # the ONNX version to export the model to\n", " do_constant_folding=True, # whether to execute constant folding for optimization\n", " input_names = ['input'], # the model's input names\n", - " output_names = ['output']) # the model's output names\n", - " #dynamic_axes={'input' : {0 : 'batch_size'}, # variable lenght axes\n", - " # 'output' : {0 : 'batch_size'}})" + " output_names = ['output']) # the model's output names\n" ] }, { @@ -288,7 +286,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "As we can not upload full calibration data set for copy right issue, we only demenstrate with some example images. You need to use your own calibration in practice." + "As we can not upload full calibration data set for copy right issue, we only demonstrate with some example images. You need to use your own calibration data set in practice." ] }, { @@ -302,11 +300,11 @@ "dr = MobilenetDataReader(calibration_data_folder)\n", "\n", "quantize_static('mobilenet_v2_float.onnx',\n", - " 'mobilenet_v2.uint8.onnx',\n", + " 'mobilenet_v2_uint8.onnx',\n", " dr)\n", "\n", "print('ONNX full precision model size (MB):', os.path.getsize(\"mobilenet_v2_float.onnx\")/(1024*1024))\n", - "print('ONNX quantized model size (MB):', os.path.getsize(\"mobilenet_v2.uint8.onnx\")/(1024*1024))" + "print('ONNX quantized model size (MB):', os.path.getsize(\"mobilenet_v2_uint8.onnx\")/(1024*1024))" ] }, { @@ -322,9 +320,49 @@ "metadata": {}, "outputs": [], "source": [ - "session_quant = onnxruntime.InferenceSession(\"mobilenet_v2.uint8.onnx\")\n", + "session_quant = onnxruntime.InferenceSession(\"mobilenet_v2_uint8.onnx\")\n", "run_sample(session_quant, 'cat.jpg', categories)" ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# 3 Convert the models to ORT format" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This step is optional, we will convert the `mobilenet_v2_float.onnx` and `mobilenet_v2_uint8.onnx` to ORT format, to be used in mobile applications.\n", + "\n", + "If you intend to run these models using ONNXRuntime Mobile Execution Providers such as [NNAPI Execution Provider](https://www.onnxruntime.ai/docs/reference/execution-providers/NNAPI-ExecutionProvider.html) or [CoreML Execution Provider](https://www.onnxruntime.ai/docs/reference/execution-providers/CoreML-ExecutionProvider.html), please set the `optimization_level` of the conversion to `basic`. If you intend to run these models using CPU only, please set the `optimization_level` of the conversion to `all`. \n", + "\n", + "For further details, please see [Converting ONNX models to ORT format](https://www.onnxruntime.ai/docs/how-to/mobile/model-conversion.html)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!{sys.executable} -m onnxruntime.tools.convert_onnx_models_to_ort --optimization_level basic ./" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Please find the following converted models in the same directory,\n", + "* mobilenet_v2_float.ort\n", + "* mobilenet_v2_uint8.ort\n", + "\n", + "The above models are used in [ONNX Runtime Mobile image classification Android sample application](https://github.com/microsoft/onnxruntime-inference-examples/tree/gwang-msft/update_mobile_example/mobile/examples/image_classifications/android).\n", + "\n", + "Please note, there are temporary ONNX model files generated by the quantization process, which are converted to ORT format as well, please ignore these files." + ] } ], "metadata": { @@ -344,6 +382,35 @@ "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.6" + }, + "varInspector": { + "cols": { + "lenName": 16, + "lenType": 16, + "lenVar": 40 + }, + "kernels_config": { + "python": { + "delete_cmd_postfix": "", + "delete_cmd_prefix": "del ", + "library": "var_list.py", + "varRefreshCmd": "print(var_dic_list())" + }, + "r": { + "delete_cmd_postfix": ") ", + "delete_cmd_prefix": "rm(", + "library": "var_list.r", + "varRefreshCmd": "cat(var_dic_list()) " + } + }, + "types_to_exclude": [ + "module", + "function", + "builtin_function_or_method", + "instance", + "_Feature" + ], + "window_display": false } }, "nbformat": 4,