saymrwulf/transformers
1
0
Fork 0
mirror of https://github.com/saymrwulf/transformers.git synced 2026-05-14 20:58:08 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Added description of quantization_config (#31133)

Browse source 
* Description of quantization_config

Added missing description about quantization_config in replace_with_bnb_linear for better readability.

* Removed trailing spaces
This commit is contained in:
Vallepu Vamsi Krishna 2024-05-31 21:53:11 +05:30 committed by GitHub
parent cdc813113a
commit 372baec2e6
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 4 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
src/transformers/integrations/bitsandbytes.py
View file

@ -243,6 +243,10 @@ def replace_with_bnb_linear(model, modules_to_not_convert=None, current_key_name
An array to track the current key of the recursion. This is used to check whether the current key (part of
it) is not in the list of modules to not convert (for instances modules that are offloaded to `cpu` or
`disk`).
quantization_config ('transformers.utils.quantization_config.BitsAndBytesConfig'):
To configure and manage settings related to quantization, a technique used to compress neural network models
by reducing the precision of the weights and activations, thus making models more efficient in terms of both
storage and computation.
"""
modules_to_not_convert = ["lm_head"] if modules_to_not_convert is None else modules_to_not_convert
model, has_been_replaced = _replace_with_bnb_linear(