Hannan Komari 8ed635258c Fix flax whisper tokenizer bug (#33151) ... * Update tokenization_whisper.py Fix issue with flax whisper model * Update tokenization_whisper_fast.py Fix issue with flax whisper model * Update tokenization_whisper.py just check len of token_ids * Update tokenization_whisper_fast.py just use len of token_ids * Update tokenization_whisper_fast.py and revert changes in _strip_prompt and add support to jax arrays in _convert_to_list * Update tokenization_whisper.py and revert changes in _strip_prompt and add support to jax arrays in _convert_to_list * Update test_tokenization_whisper.py to add test for _convert_to_list method * Update test_tokenization_whisper.py to fix code style issues * Fix code style * Fix code check again * Update test_tokenization)whisper.py to Improve code style * Update test_tokenization_whisper.py to run each of jax, tf and flax modules if available * Update tests/models/whisper/test_tokenization_whisper.py Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com> * Update test_tokenization_whisper.py and use require_xxx decorators instead of `is_xxx_available()` method * Revert the changes automatically applied by formatter and was unrelated to PR * Format for minimal changes --------- Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>		2024-09-12 12:21:59 +01:00
..
agents	Multi agents with manager (#32687)	2024-09-04 17:30:54 +02:00
benchmark
bettertransformer
deepspeed	Revert PR 32299, flag users when Zero-3 was missed (#32851)	2024-08-16 12:35:41 -04:00
extended	Skip tests properly (#31308)	2024-06-26 21:59:08 +01:00
fixtures
fsdp	🚨🚨🚨 Update min version of accelerate to 0.26.0 (#32627)	2024-08-20 11:42:36 +02:00
generation	Dynamic number of speculative tokens in order to accelerate speculative decoding (#33258)	2024-09-11 14:22:28 +02:00
models	Fix flax whisper tokenizer bug (#33151)	2024-09-12 12:21:59 +01:00
optimization	fix: Fixed the 1st argument name in classmethods (#31907)	2024-07-11 12:11:50 +01:00
peft_integration
pipelines	Compile compatibilty for decoder-only models (#32617)	2024-09-09 10:59:04 +02:00
quantization	Fix FbgemmFp8Linear not preserving tensor shape (#33239)	2024-09-11 13:26:44 +02:00
repo_utils	Refactor CI: more explicit (#30674)	2024-08-30 18:17:25 +02:00
sagemaker	Fixed log messages that are resulting in TypeError due to too many arguments (#32017)	2024-07-17 10:56:44 +01:00
tokenization	#32184 save total_vocab_size (#32240)	2024-08-05 09:22:48 +02:00
trainer	schedulefree optimizers (#30079)	2024-09-09 09:51:39 +02:00
utils	Make StaticCache configurable at model construct time (#32830)	2024-09-10 16:35:57 +01:00
__init__.py
test_backbone_common.py
test_configuration_common.py	Refactor: Removed un-necessary object base class (#32230)	2024-07-26 10:33:02 +02:00
test_feature_extraction_common.py
test_image_processing_common.py	Update kwargs validation for preprocess with decorator (#32024)	2024-08-06 11:33:05 +01:00
test_image_transforms.py
test_modeling_common.py	Compile compatibilty for decoder-only models (#32617)	2024-09-09 10:59:04 +02:00
test_modeling_flax_common.py
test_modeling_tf_common.py
test_pipeline_mixin.py	Config: unified logic to retrieve text config (#33219)	2024-09-04 12:03:30 +01:00
test_processing_common.py	Remove repeated prepare_images in processor tests (#33163)	2024-09-09 13:20:27 +01:00
test_sequence_feature_extraction_common.py
test_tokenization_common.py	Add assistant prefill for chat templates and TextGenerationPipeline (#33198)	2024-09-02 13:23:47 +01:00