saymrwulf/onnxruntime
1
0
Fork 0
mirror of https://github.com/saymrwulf/onnxruntime.git synced 2026-07-30 20:18:08 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Minor updates to include QDQ Dynamic (#12839)

Browse source 
Minor updates in the ONNX quantization representation format section to reflect the addition of QDQ Dynamic feature.
This commit is contained in:
acid-space-cowboy 2022-09-02 11:18:36 -07:00 committed by GitHub
parent f5b30bdfc9
commit c0e3e10f34
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 2 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
docs/performance/quantization.md
View file

@ -36,8 +36,8 @@ Quantization in ONNX Runtime refers to 8 bit linear quantization of an ONNX mode
## ONNX quantization representation format
There are two ways to represent quantized ONNX models:
- Operator-oriented (QOperator). All the quantized operators have their own ONNX definitions, like QLinearConv, MatMulInteger and etc.
- Tensor-oriented (QDQ; Quantize and DeQuantize). This format inserts DeQuantizeLinear(QuantizeLinear(tensor)) between the original operators to simulate the quantization and dequantization process. The QuantizeLinear and DeQuantizeLinear operators also carry the quantization parameters.Models generated in the following ways are in the QDQ format:
- Models quantized by quantize_static API below with `quant_format=QuantFormat.QDQ`.
- Tensor-oriented (QDQ; Quantize and DeQuantize). This format inserts DeQuantizeLinear(QuantizeLinear(tensor)) between the original operators to simulate the quantization and dequantization process. In Static Quantization, the QuantizeLinear and DeQuantizeLinear operators also carry the quantization parameters. In Dynamic Quantization, a ComputeQuantizationParameters function proto is inserted to calculate quantization parameters on the fly. Models generated in the following ways are in the QDQ format:
- Models quantized by quantize_static or quantize_dynamic API, explained below, with `quant_format=QuantFormat.QDQ`.
- Quantization-Aware training (QAT) models converted from Tensorflow or exported from PyTorch.
- Quantized models converted from TFLite and other frameworks.