From 8533e50b24f30e2a0f613a261c30bf4528da35ad Mon Sep 17 00:00:00 2001 From: Ender Date: Tue, 17 May 2022 02:08:32 +0800 Subject: [PATCH] update docs/performance: fix typo (#11372) Here it should be `max` instead of `abs` for symmetric quantization. --- docs/performance/quantization.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/performance/quantization.md b/docs/performance/quantization.md index 7c325a2a25..fe59cca988 100644 --- a/docs/performance/quantization.md +++ b/docs/performance/quantization.md @@ -28,7 +28,7 @@ Quantization in ONNX Runtime refers to 8 bit linear quantization of an ONNX mode For symmetric quantization: ``` - scale = abs(data_range_max, data_range_min) * 2 / (quantization_range_max - quantization_range_min) + scale = max(abs(data_range_max), abs(data_range_min)) * 2 / (quantization_range_max - quantization_range_min) ``` Zero_point represents zero in the quantization space. It is important that the floating point zero value be exactly representable in quantization space. This is because zero padding is used in many CNNs. If it is not possible to represent 0 uniquely after quantization, it will result in accuracy errors.