mirror of
https://github.com/saymrwulf/onnxruntime.git
synced 2026-07-30 20:18:08 +00:00
Update BERT model optimization python script (#2521)
Add support of GPT2 model optimization: * Match subgraph of Gelu Approximation (using Tanh). * Fuse LayerNormalization if SkipLayerNormalization is not ready. * Output model even if embedding layer is not fused. * Improve Reshape Fusion to improve coverage. * Refine constant input checking, and output fused op counter. Update script according to latest op improvements: * Fusion of Add Bias and Gelu. * Fuse SkipLayerNormalization and Add Bias. Other: * Add ReduceSum for mask as intermediate step. * Refactor verbose setting.
This commit is contained in:
parent
31ea11a696
commit
66254eb25a
2 changed files with 550 additions and 170 deletions
|
|
@ -52,4 +52,7 @@ See below for description of all the options:
|
|||
- **input_int32**: (*optional*)
|
||||
Exported model ususally uses int64 tensor as input. If this flag is specified, int32 tensors will be used as input, and it could avoid un-necessary Cast nodes and get better performance.
|
||||
- **float16**: (*optional*)
|
||||
By default, model uses float32 in computation. If this flag is specified, half-precision float will be used. This option is recommended for NVidia GPU with Tensor Core like V100 and T4. For older GPUs, float32 is likely faster.
|
||||
By default, model uses float32 in computation. If this flag is specified, half-precision float will be used. This option is recommended for NVidia GPU with Tensor Core like V100 and T4. For older GPUs, float32 is likely faster.
|
||||
- **verbose**: (*optional*)
|
||||
Print verbose information when this flag is specified.
|
||||
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
Loading…
Reference in a new issue