Add Notebook for Bert Model exported by Keras2onnx (#3271)

* Add notebook for bert squad model exported by python 1.4

* update bert performance test tool:
(1) set OpenMP environment variable before importing onnxruntime.
(2) launch new process for each test.

* Add notebook
Reduce combinations in perf test

* update readme

* fix quote

* Allow test multiple batch_size

* Add latency percentile

* Add warm up run
Reset logger for notebook

* refine default settings to test for cpu/gpu

* Add script to dump machine info

* Add notebooks for PyTorch SQuAD model GPU and CPU inference

* Update machineinfo.py: add license header; format by yapf

* Do not reset log handler. Skip adding handler if existed.

* Add comments about GPU result diff.
Filter rows of batch set to keep only one setting.

* update according to review feedback

* Download script from master branch

* Add notebook for bert model exported by keras2onnx

* format columns in result table

* re-run and update notebook

onnxruntime/python/tools/bert/bert_model_optimization.py

@@ -30,7 +30,6 @@ import argparse
 import numpy as np
 from collections import deque
 from onnx import ModelProto, TensorProto, numpy_helper
-import onnxruntime
 from BertOnnxModel import BertOnnxModel
 from BertOnnxModelTF import BertOnnxModelTF
 from BertOnnxModelKeras import BertOnnxModelKeras
@@ -56,6 +55,8 @@ def optimize_by_onnxruntime(onnx_model_path, use_gpu, optimized_model_path=None)
     Returns:
         optimized_model_path: the path of optimized model
     """
+    import onnxruntime
+
     if use_gpu and 'CUDAExecutionProvider' not in onnxruntime.get_available_providers():
         logger.error("There is no gpu for onnxruntime to do optimization.")
         return onnx_model_path