saymrwulf/onnxruntime
1
0
Fork 0
mirror of https://github.com/saymrwulf/onnxruntime.git synced 2026-07-30 20:18:08 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Fix quant_format argument for 4bit quantizer (#21581)

Browse source 
### Description
Original argument accepts Enum QuantFormat.QOperator or QuantFormat.QDQ,
but the default value is QOperator.

Change the argument to str to accept QOperator or QDQ and convert to
QuantFormat after parsing.

### Motivation and Context
Bug fix
This commit is contained in:
Jing Fang 2024-07-31 15:30:33 -07:00 committed by GitHub
parent a3883af7bf
commit 8540ac4f78
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 3 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
onnxruntime/python/tools/quantization/matmul_4bits_quantizer.py
View file

@ -797,8 +797,8 @@ set of 4b integers with a scaling factor and an optional offset.
parser.add_argument(
"--quant_format",
default="QOperator",
type=QuantFormat,
choices=list(QuantFormat),
type=str,
choices=["QOperator", "QDQ"],
help="QuantFormat {QOperator, QDQ}"
"QOperator format quantizes the model with quantized operators directly."
"QDQ format quantize the model by inserting DeQuantizeLinear before the MatMul.",
@ -814,7 +814,7 @@ if __name__ == "__main__":
input_model_path = args.input_model
output_model_path = args.output_model
quant_format = args.quant_format
quant_format = QuantFormat[args.quant_format]
if os.path.exists(output_model_path):
logger.error(f"file {output_model_path} already exists")