Raushan Turganbay 5ad960f1f4 Add Watermarking LogitsProcessor and WatermarkDetector (#29676) ... * add watermarking processor * remove the other hashing (context width=1 always) * make style * Update src/transformers/generation/logits_process.py Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com> * Update src/transformers/generation/logits_process.py Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com> * Update src/transformers/generation/logits_process.py Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com> * Update src/transformers/generation/configuration_utils.py Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com> * update watermarking process * add detector * update tests to use detector * fix failing tests * rename `input_seq` * make style * doc for processor * minor fixes * docs * make quality * Update src/transformers/generation/configuration_utils.py Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com> * Update src/transformers/generation/logits_process.py Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com> * Update src/transformers/generation/watermarking.py Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com> * Update src/transformers/generation/watermarking.py Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com> * Update src/transformers/generation/watermarking.py Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com> * add PR suggestions * let's use lru_cache's default max size (128) * import processor if torch available * maybe like this * lets move the config to torch independet file * add docs * tiny docs fix to make the test happy * Update src/transformers/generation/configuration_utils.py Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com> * Update src/transformers/generation/watermarking.py Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com> * PR suggestions * add docs * fix test * fix docs * address pr comments * style * Revert "style" This reverts commit 7f33cc34ff08b414f8e7f90060889877606b43b2. * correct style * make doctest green --------- Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>		2024-05-14 13:31:39 +05:00
..
__init__.py
test_beam_constraints.py
test_beam_search.py
test_configuration_utils.py	Generate: get generation mode from the generation config instance 🧼 (#29441)	2024-03-06 11:18:35 +00:00
test_flax_logits_process.py	Adding FlaxNoRepeatNGramLogitsProcessor (#29677)	2024-04-02 11:39:33 +02:00
test_flax_utils.py
test_framework_agnostic.py	Revert workaround for TF safetensors loading (#30128)	2024-04-09 11:04:18 +01:00
test_logits_process.py	Add Watermarking LogitsProcessor and WatermarkDetector (#29676)	2024-05-14 13:31:39 +05:00
test_stopping_criteria.py	Terminator strings for generate() (#28932)	2024-04-22 14:13:04 +01:00
test_streamers.py	Update all references to canonical models (#29001)	2024-02-16 08:16:58 +01:00
test_tf_logits_process.py
test_tf_utils.py	Revert workaround for TF safetensors loading (#30128)	2024-04-09 11:04:18 +01:00
test_utils.py	Add Watermarking LogitsProcessor and WatermarkDetector (#29676)	2024-05-14 13:31:39 +05:00