saymrwulf/transformers
1
0
Fork 0
mirror of https://github.com/saymrwulf/transformers.git synced 2026-05-14 20:58:08 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
transformers/tests
History
Arthur b52a03cd3b
⚠️⚠️[T5Tokenize] Fix T5 family tokenizers⚠️⚠️ (#24565) 
* don't add space before single letter chars that don't have a merge

* fix the fix

* fixup

* add a test

* more testing

* fixup

* hack to make sure fast is also fixed

* update switch transformers test

* revert convert slow

* Update src/transformers/models/t5/tokenization_t5.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* add typechecking

* quality

---------

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-06-30 07:00:43 +02:00
..
benchmark
bettertransformer
bnb [gpt2-int8] Add gpt2-xl int8 test (#24543) 2023-06-28 18:02:13 +02:00
deepspeed accelerate deepspeed and gradient accumulation integrate (#23236) 2023-05-31 15:16:22 +05:30
extended
fixtures
generation Fix TypeError: Object of type int64 is not JSON serializable (#24340) 2023-06-27 12:15:49 +01:00
models ⚠️⚠️[T5Tokenize] Fix T5 family tokenizers⚠️⚠️ (#24565) 2023-06-30 07:00:43 +02:00
onnx Fix TypeError: Object of type int64 is not JSON serializable (#24340) 2023-06-27 12:15:49 +01:00
optimization
pipelines [pipeline] Fix str device issue (#24396) 2023-06-26 13:58:36 +02:00
repo_utils Fix expected value in tests of the test fetcher (#24077) 2023-06-07 11:38:56 -04:00
sagemaker
tokenization
tools Add support for for loops in python interpreter (#24429) 2023-06-26 09:58:14 -04:00
trainer Fix TypeError: Object of type int64 is not JSON serializable (#24340) 2023-06-27 12:15:49 +01:00
utils ⚠️ Time to say goodbye to py37 (#24091) 2023-06-28 07:22:39 +02:00
__init__.py
test_backbone_common.py Add TimmBackbone model (#22619) 2023-06-06 17:11:30 +01:00
test_configuration_common.py Split common test from core tests (#24284) 2023-06-15 07:30:24 -04:00
test_configuration_utils.py Split common test from core tests (#24284) 2023-06-15 07:30:24 -04:00
test_feature_extraction_common.py Split common test from core tests (#24284) 2023-06-15 07:30:24 -04:00
test_feature_extraction_utils.py Split common test from core tests (#24284) 2023-06-15 07:30:24 -04:00
test_image_processing_common.py Split common test from core tests (#24284) 2023-06-15 07:30:24 -04:00
test_image_processing_utils.py Split common test from core tests (#24284) 2023-06-15 07:30:24 -04:00
test_image_transforms.py Bug fix - flip_channel_order for channels first images (#23701) 2023-05-31 17:12:27 +01:00
test_modeling_common.py Make PT/Flax tests could be run on GPU (#24557) 2023-06-28 20:11:01 +02:00
test_modeling_flax_common.py Split common test from core tests (#24284) 2023-06-15 07:30:24 -04:00
test_modeling_flax_utils.py Split common test from core tests (#24284) 2023-06-15 07:30:24 -04:00
test_modeling_tf_common.py Fix TypeError: Object of type int64 is not JSON serializable (#24340) 2023-06-27 12:15:49 +01:00
test_modeling_tf_utils.py Split common test from core tests (#24284) 2023-06-15 07:30:24 -04:00
test_modeling_utils.py Clean load keys (#24505) 2023-06-27 14:45:40 -04:00
test_pipeline_mixin.py Update tiny models for pipeline testing. (#24364) 2023-06-20 14:43:10 +02:00
test_sequence_feature_extraction_common.py
test_tokenization_common.py Fix TypeError: Object of type int64 is not JSON serializable (#24340) 2023-06-27 12:15:49 +01:00
test_tokenization_utils.py Split common test from core tests (#24284) 2023-06-15 07:30:24 -04:00