Update BERT model optimization python script (#2521)

Add support of GPT2 model optimization:
* Match subgraph of Gelu Approximation (using Tanh).
* Fuse LayerNormalization if SkipLayerNormalization is not ready.
* Output model even if embedding layer is not fused.
* Improve Reshape Fusion to improve coverage.
* Refine constant input checking, and output fused op counter.

Update script according to latest op improvements:
* Fusion of Add Bias and Gelu.
* Fuse SkipLayerNormalization and Add Bias.

Other:
* Add ReduceSum for mask as intermediate step.
* Refactor verbose setting.
This commit is contained in:
Tianlei Wu 2019-12-03 08:40:51 -08:00 committed by GitHub
parent 31ea11a696
commit 66254eb25a
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
2 changed files with 550 additions and 170 deletions

View file

@ -52,4 +52,7 @@ See below for description of all the options:
- **input_int32**: (*optional*)
Exported model ususally uses int64 tensor as input. If this flag is specified, int32 tensors will be used as input, and it could avoid un-necessary Cast nodes and get better performance.
- **float16**: (*optional*)
By default, model uses float32 in computation. If this flag is specified, half-precision float will be used. This option is recommended for NVidia GPU with Tensor Core like V100 and T4. For older GPUs, float32 is likely faster.
By default, model uses float32 in computation. If this flag is specified, half-precision float will be used. This option is recommended for NVidia GPU with Tensor Core like V100 and T4. For older GPUs, float32 is likely faster.
- **verbose**: (*optional*)
Print verbose information when this flag is specified.

File diff suppressed because it is too large Load diff