mirror of
https://github.com/saymrwulf/onnxruntime.git
synced 2026-05-16 21:00:14 +00:00
fe463d4957
### Description Support SmoothQuant for ORT static quantization via intel neural compressor > Note: Please use neural-compressor==2.2 to try SmoothQuant function. ### Motivation and Context For large language models (LLMs) with gigantic parameters, the systematic outliers make quantification of activations difficult. As a training free post-training quantization (PTQ) solution, SmoothQuant offline migrates this difficulty from activations to weights with a mathematically equivalent transformation. Integrating SmoothQuant into ORT quantization can benefit the accuracy of INT8 LLMs. --------- Signed-off-by: Mengni Wang <mengni.wang@intel.com> |
||
|---|---|---|
| .. | ||
| docker | ||
| ort_minimal | ||
| tvm | ||
| build_cuda_c_api_package.sh | ||
| build_linux_arm64_python_package.sh | ||
| build_yocto.sh | ||
| copy_strip_binary.sh | ||
| create_package.sh | ||
| extract_and_bundle_gpu_package.sh | ||
| java_copy_strip_binary.sh | ||
| java_linux_final_test.sh | ||
| run_build.sh | ||
| run_dockerbuild.sh | ||
| run_python_dockerbuild.sh | ||
| run_python_tests.sh | ||
| test_custom_ops_pytorch_export.sh | ||
| upload_code_coverage_data.sh | ||
| upload_ortsrv_binaries.sh | ||
| yocto_build_toolchain.sh | ||