saymrwulf/onnxruntime
1
0
Fork 0
mirror of https://github.com/saymrwulf/onnxruntime.git synced 2026-07-29 20:14:01 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Refine op_types_to_quantize argument handling in matmul_4bits_quantizer.py (#21815)

Browse source 
### Description
<!-- Describe your changes. -->

Refine `op_types_to_quantize` argument handling in
matmul_4bits_quantizer.py

### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
The default `op_types_to_quantize "MatMul"` will cause
`tuple(args.op_types_to_quantize)` to become `('M', 'a', 't', 'M', 'u',
'l')`, which is not expected.
This commit is contained in:
duanshengliu 2024-08-24 04:45:06 +08:00 committed by GitHub
parent 44dcc3aafd
commit 4af6291841
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 1 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
onnxruntime/python/tools/quantization/matmul_4bits_quantizer.py
View file

@ -1062,7 +1062,6 @@ set of 4b integers with a scaling factor and an optional offset.
)
parser.add_argument(
"--op_types_to_quantize",
default="MatMul",
type=str,
nargs="+",
choices=["MatMul", "Gather"],
@ -1089,7 +1088,7 @@ if __name__ == "__main__":
input_model_path = args.input_model
output_model_path = args.output_model
quant_format = QuantFormat[args.quant_format]
op_types_to_quantize = tuple(args.op_types_to_quantize) if args.op_types_to_quantize else None
op_types_to_quantize = tuple(args.op_types_to_quantize) if args.op_types_to_quantize else ("MatMul",)
quant_axes = tuple(args.quant_axes) if args.quant_axes else None
if os.path.exists(output_model_path):