saymrwulf/onnxruntime
1
0
Fork 0
mirror of https://github.com/saymrwulf/onnxruntime.git synced 2026-07-23 19:32:23 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Update mobilenetv2 quantization notebook (#7941)

Browse source 
* Update MobileNetV2 notebook for mobile

* Remove outputs of the notebook

* minor update

* Address CR comments

* update comments of the notebook
This commit is contained in:
Guoyu Wang 2021-06-04 18:15:10 -07:00 committed by GitHub
parent 291453dac9
commit fd23b8caad
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 81 additions and 14 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

95
onnxruntime/python/tools/quantization/notebooks/imagenet_v2/mobilenet.ipynb
View file

@ -19,7 +19,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"In this tutorial, we will load a mobilenet v2 model pretrained with [PyTorch](https://pytorch.org/), export the model to ONNX, and then quantize and run with ONNXRuntime."
"In this tutorial, we will load a mobilenet v2 model pretrained with [PyTorch](https://pytorch.org/), export the model to ONNX, quantize then run with ONNXRuntime, and convert the ONNX models to ORT format for ONNXRuntime Mobile."
]
},
{
@ -30,7 +30,7 @@
"\n",
"If you have Jupyter Notebook, you can run this notebook directly with it. You may need to install or upgrade [PyTorch](https://pytorch.org/), [OnnxRuntime](https://microsoft.github.io/onnxruntime/), and other required packages.\n",
"\n",
"Otherwise, you can setup a new environment. First, install [AnaConda](https://www.anaconda.com/distribution/). Then open an AnaConda prompt window and run the following commands:\n",
"Otherwise, you can setup a new environment. First, install [Anaconda](https://www.anaconda.com/distribution/). Then open an AnaConda prompt window and run the following commands:\n",
"\n",
"```console\n",
"conda create -n cpu_env python=3.8\n",
@ -46,7 +46,7 @@
"metadata": {},
"source": [
"### 0.1 Install packages\n",
"Let's install nessasary packages to start the tutorial. We will install PyTorch 1.8, OnnxRuntime 1.7, latest ONNX and pillow."
"Let's install the necessary packages to start the tutorial. We will install PyTorch 1.8, OnnxRuntime 1.8, latest ONNX and pillow."
]
},
{
@ -57,10 +57,10 @@
},
"outputs": [],
"source": [
"# Install or upgrade PyTorch 1.8.0 and OnnxRuntime 1.7 for CPU-only.\n",
"# Install or upgrade PyTorch 1.8.0 and OnnxRuntime 1.8 for CPU-only.\n",
"import sys\n",
"!{sys.executable} -m pip install --upgrade torch==1.8.0+cpu torchvision==0.9.0+cpu torchaudio===0.8.0 -f https://download.pytorch.org/whl/torch_stable.html\n",
"!{sys.executable} -m pip install --upgrade onnxruntime==1.7.0\n",
"!{sys.executable} -m pip install --upgrade torch==1.8.0 torchvision==0.9.0 torchaudio===0.8.0 -f https://download.pytorch.org/whl/torch_stable.html\n",
"!{sys.executable} -m pip install --upgrade onnxruntime==1.8.0\n",
"!{sys.executable} -m pip install --upgrade onnx\n",
"!{sys.executable} -m pip install --upgrade pillow"
]
@ -120,14 +120,12 @@
"# Export the model\n",
"torch.onnx.export(mobilenet_v2, # model being run\n",
" x, # model input (or a tuple for multiple inputs)\n",
" \"mobilenet_v2_float.onnx\", # where to save the model (can be a file or file-like object)\n",
" \"mobilenet_v2_float.onnx\", # where to save the model (can be a file or file-like object)\n",
" export_params=True, # store the trained parameter weights inside the model file\n",
" opset_version=12, # the ONNX version to export the model to\n",
" do_constant_folding=True, # whether to execute constant folding for optimization\n",
" input_names = ['input'], # the model's input names\n",
" output_names = ['output']) # the model's output names\n",
" #dynamic_axes={'input' : {0 : 'batch_size'}, # variable lenght axes\n",
" # 'output' : {0 : 'batch_size'}})"
" output_names = ['output']) # the model's output names\n"
]
},
{
@ -288,7 +286,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"As we can not upload full calibration data set for copy right issue, we only demenstrate with some example images. You need to use your own calibration in practice."
"As we can not upload full calibration data set for copy right issue, we only demonstrate with some example images. You need to use your own calibration data set in practice."
]
},
{
@ -302,11 +300,11 @@
"dr = MobilenetDataReader(calibration_data_folder)\n",
"\n",
"quantize_static('mobilenet_v2_float.onnx',\n",
" 'mobilenet_v2.uint8.onnx',\n",
" 'mobilenet_v2_uint8.onnx',\n",
" dr)\n",
"\n",
"print('ONNX full precision model size (MB):', os.path.getsize(\"mobilenet_v2_float.onnx\")/(1024*1024))\n",
"print('ONNX quantized model size (MB):', os.path.getsize(\"mobilenet_v2.uint8.onnx\")/(1024*1024))"
"print('ONNX quantized model size (MB):', os.path.getsize(\"mobilenet_v2_uint8.onnx\")/(1024*1024))"
]
},
{
@ -322,9 +320,49 @@
"metadata": {},
"outputs": [],
"source": [
"session_quant = onnxruntime.InferenceSession(\"mobilenet_v2.uint8.onnx\")\n",
"session_quant = onnxruntime.InferenceSession(\"mobilenet_v2_uint8.onnx\")\n",
"run_sample(session_quant, 'cat.jpg', categories)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 3 Convert the models to ORT format"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This step is optional, we will convert the `mobilenet_v2_float.onnx` and `mobilenet_v2_uint8.onnx` to ORT format, to be used in mobile applications.\n",
"\n",
"If you intend to run these models using ONNXRuntime Mobile Execution Providers such as [NNAPI Execution Provider](https://www.onnxruntime.ai/docs/reference/execution-providers/NNAPI-ExecutionProvider.html) or [CoreML Execution Provider](https://www.onnxruntime.ai/docs/reference/execution-providers/CoreML-ExecutionProvider.html), please set the `optimization_level` of the conversion to `basic`. If you intend to run these models using CPU only, please set the `optimization_level` of the conversion to `all`. \n",
"\n",
"For further details, please see [Converting ONNX models to ORT format](https://www.onnxruntime.ai/docs/how-to/mobile/model-conversion.html)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!{sys.executable} -m onnxruntime.tools.convert_onnx_models_to_ort --optimization_level basic ./"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Please find the following converted models in the same directory,\n",
"* mobilenet_v2_float.ort\n",
"* mobilenet_v2_uint8.ort\n",
"\n",
"The above models are used in [ONNX Runtime Mobile image classification Android sample application](https://github.com/microsoft/onnxruntime-inference-examples/tree/gwang-msft/update_mobile_example/mobile/examples/image_classifications/android).\n",
"\n",
"Please note, there are temporary ONNX model files generated by the quantization process, which are converted to ORT format as well, please ignore these files."
]
}
],
"metadata": {
@ -344,6 +382,35 @@
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.6"
},
"varInspector": {
"cols": {
"lenName": 16,
"lenType": 16,
"lenVar": 40
},
"kernels_config": {
"python": {
"delete_cmd_postfix": "",
"delete_cmd_prefix": "del ",
"library": "var_list.py",
"varRefreshCmd": "print(var_dic_list())"
},
"r": {
"delete_cmd_postfix": ") ",
"delete_cmd_prefix": "rm(",
"library": "var_list.r",
"varRefreshCmd": "cat(var_dic_list()) "
}
},
"types_to_exclude": [
"module",
"function",
"builtin_function_or_method",
"instance",
"_Feature"
],
"window_display": false
}
},
"nbformat": 4,