saymrwulf/transformers
1
0
Fork 0
mirror of https://github.com/saymrwulf/transformers.git synced 2026-05-14 20:58:08 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
transformers/tests/models
History
Lysandre Debut 113ebf80ac
Safetensors serialization by default (#27064) 
* Safetensors serialization by default

* First pass on the tests

* Second pass on the tests

* Third pass on the tests

* Fix TF weight loading from TF-format safetensors

* Specific encoder-decoder fixes for weight crossloading

* Add VisionEncoderDecoder fixes for TF too

* Change filename test for pt-to-tf

* One missing fix for TFVisionEncoderDecoder

* Fix the other crossload test

* Support for flax + updated tests

* Apply suggestions from code review

Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>

* Sanchit's comments

* Sanchit's comments 2

* Nico's comments

* Fix tests

* cleanup

* Apply suggestions from code review

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

---------

Co-authored-by: Matt <rocketknight1@gmail.com>
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
2023-10-31 19:16:49 +01:00
..
albert CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
align [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
altclip [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
audio_spectrogram_transformer Add # Copied from statements to audio feature extractors that use the floats_list function (#26581) 2023-10-04 17:09:48 +02:00
auto Safetensors serialization by default (#27064) 2023-10-31 19:16:49 +01:00
autoformer [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
bark device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
bart device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
barthez
bartpho
beit [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
bert device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
bert_generation CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
bert_japanese
bertweet
big_bird [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
bigbird_pegasus device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
biogpt Copied from for test files (#26713) 2023-10-11 14:12:09 +02:00
bit
blenderbot device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
blenderbot_small device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
blip device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
blip_2 device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
bloom Device agnostic testing (#25870) 2023-10-24 16:49:26 +02:00
bridgetower Refactor image processor testers (#25450) 2023-08-11 11:30:18 +01:00
bros Update tiny model information and pipeline tests (#26285) 2023-09-25 18:08:12 +02:00
byt5 🚨🚨 🚨🚨 [Tokenizer] attemp to fix add_token issues🚨🚨 🚨🚨 (#23909) 2023-09-18 20:28:36 +02:00
camembert [Tokenizer] Fix slow and fast serialization (#26570) 2023-10-18 16:30:53 +02:00
canine [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
chinese_clip [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
clap [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
clip [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
clipseg [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
code_llama [Tokenizer] Fix slow and fast serialization (#26570) 2023-10-18 16:30:53 +02:00
codegen Device agnostic testing (#25870) 2023-10-24 16:49:26 +02:00
conditional_detr Refactor image processor testers (#25450) 2023-08-11 11:30:18 +01:00
convbert device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
convnext Refactor image processor testers (#25450) 2023-08-11 11:30:18 +01:00
convnextv2
cpm
cpmant CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
ctrl device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
cvt
data2vec Fix typo (#25966) 2023-09-05 10:12:25 +02:00
deberta CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
deberta_v2 🚨🚨 🚨🚨 [Tokenizer] attemp to fix add_token issues🚨🚨 🚨🚨 (#23909) 2023-09-18 20:28:36 +02:00
decision_transformer
deformable_detr device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
deit device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
deta Refactor image processor testers (#25450) 2023-08-11 11:30:18 +01:00
detr Fix Detr CI (#25972) 2023-09-05 11:19:56 +02:00
dinat
dinov2 [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
distilbert device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
dit
donut Refactor image processor testers (#25450) 2023-08-11 11:30:18 +01:00
dpr CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
dpt [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
efficientformer Refactor image processor testers (#25450) 2023-08-11 11:30:18 +01:00
efficientnet 🚨🚨🚨 Remove softmax for EfficientNetForImageClassification 🚨🚨🚨 (#25501) 2023-08-14 17:08:47 +01:00
electra CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
encodec Add # Copied from statements to audio feature extractors that use the floats_list function (#26581) 2023-10-04 17:09:48 +02:00
encoder_decoder Safetensors serialization by default (#27064) 2023-10-31 19:16:49 +01:00
ernie device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
ernie_m CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
esm added support for gradient checkpointing in ESM models (#26386) 2023-09-26 10:15:53 +02:00
falcon Fix Falcon generation test (#26770) 2023-10-13 15:10:27 +01:00
flaubert device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
flava [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
fnet [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
focalnet
fsmt device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
funnel
fuyu device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
git [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
glpn Input data format (#25464) 2023-08-16 17:45:02 +01:00
gpt2 device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
gpt_bigcode CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
gpt_neo CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
gpt_neox CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
gpt_neox_japanese CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
gpt_sw3 🚨🚨 🚨🚨 [Tokenizer] attemp to fix add_token issues🚨🚨 🚨🚨 (#23909) 2023-09-18 20:28:36 +02:00
gptj device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
gptsan_japanese 🚨🚨 🚨🚨 [Tokenizer] attemp to fix add_token issues🚨🚨 🚨🚨 (#23909) 2023-09-18 20:28:36 +02:00
graphormer
groupvit [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
herbert [Tokenizer] Fix slow and fast serialization (#26570) 2023-10-18 16:30:53 +02:00
hubert Fix failing tests on main due to torch 2.1 (#26607) 2023-10-05 10:27:05 +02:00
ibert CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
idefics [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
imagegpt [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
informer [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
instructblip [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
jukebox device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
kosmos2 Add Kosmos-2 model (#24709) 2023-10-30 13:32:17 +01:00
layoutlm [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
layoutlmv2 Overhaul Conversation class and prompt templating (#25323) 2023-09-14 15:10:34 +01:00
layoutlmv3 🚨🚨 🚨🚨 [Tokenizer] attemp to fix add_token issues🚨🚨 🚨🚨 (#23909) 2023-09-18 20:28:36 +02:00
layoutxlm Overhaul Conversation class and prompt templating (#25323) 2023-09-14 15:10:34 +01:00
led device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
levit Refactor image processor testers (#25450) 2023-08-11 11:30:18 +01:00
lilt [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
llama device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
longformer Copied from for test files (#26713) 2023-10-11 14:12:09 +02:00
longt5 CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
luke [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
lxmert
m2m_100 device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
marian device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
markuplm [Tokenizer] Fix slow and fast serialization (#26570) 2023-10-18 16:30:53 +02:00
mask2former device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
maskformer device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
mbart device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
mbart50
mega device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
megatron_bert CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
megatron_gpt2
mgp_str Refactor image processor testers (#25450) 2023-08-11 11:30:18 +01:00
mistral device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
mluke 🚨🚨 🚨🚨 [Tokenizer] attemp to fix add_token issues🚨🚨 🚨🚨 (#23909) 2023-09-18 20:28:36 +02:00
mobilebert Copied from for test files (#26713) 2023-10-11 14:12:09 +02:00
mobilenet_v1 Refactor image processor testers (#25450) 2023-08-11 11:30:18 +01:00
mobilenet_v2 Refactor image processor testers (#25450) 2023-08-11 11:30:18 +01:00
mobilevit Refactor image processor testers (#25450) 2023-08-11 11:30:18 +01:00
mobilevitv2
mpnet CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
mpt Fix test_modeling_mpt typo in model id (#25606) 2023-08-21 11:11:21 +02:00
mra [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
mt5
musicgen device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
mvp device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
nat
nezha CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
nllb [Tokenizers] Skip tests temporarily (#26574) 2023-10-03 19:43:42 +02:00
nllb_moe device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
nougat Add Nougat (#25942) 2023-09-26 07:06:04 +02:00
nystromformer CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
oneformer device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
openai CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
opt device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
owlv2 device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
owlvit device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
pegasus device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
pegasus_x device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
perceiver Fix PerceiverModelIntegrationTest::test_inference_masked_lm (#26760) 2023-10-12 17:43:06 +02:00
persimmon device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
phobert
pix2struct [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
plbart device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
poolformer Refactor image processor testers (#25450) 2023-08-11 11:30:18 +01:00
pop2piano Update tiny model information and pipeline tests (#26285) 2023-09-25 18:08:12 +02:00
prophetnet CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
pvt device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
qdqbert CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
rag
realm CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
reformer Device agnostic testing (#25870) 2023-10-24 16:49:26 +02:00
regnet
rembert CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
resnet
roberta Copied from for test files (#26713) 2023-10-11 14:12:09 +02:00
roberta_prelayernorm Copied from for test files (#26713) 2023-10-11 14:12:09 +02:00
roc_bert Copied from for test files (#26713) 2023-10-11 14:12:09 +02:00
roformer [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
rwkv CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
sam device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
seamless_m4t [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
segformer Refactor image processor testers (#25450) 2023-08-11 11:30:18 +01:00
sew Fix typo (#25966) 2023-09-05 10:12:25 +02:00
sew_d Fix typo (#25966) 2023-09-05 10:12:25 +02:00
speech_encoder_decoder
speech_to_text device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
speech_to_text_2 CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
speecht5 [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
splinter CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
squeezebert CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
swiftformer Fix last models for common tests that are too big. (#25058) 2023-07-25 07:56:04 -04:00
swin
swin2sr [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
swinv2
switch_transformers device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
t5 [Tokenizer] Fix slow and fast serialization (#26570) 2023-10-18 16:30:53 +02:00
table_transformer Fix last models for common tests that are too big. (#25058) 2023-07-25 07:56:04 -04:00
tapas Overhaul Conversation class and prompt templating (#25323) 2023-09-14 15:10:34 +01:00
time_series_transformer [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
timesformer CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
timm_backbone [AutoBackbone] Add test (#26094) 2023-09-18 23:47:54 +02:00
transfo_xl CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
trocr CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
tvlt Add # Copied from statements to audio feature extractors that use the floats_list function (#26581) 2023-10-04 17:09:48 +02:00
umt5 [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
unispeech [Wav2Vec2 and Co] Update init tests for PT 2.1 (#26494) 2023-10-03 10:52:34 +02:00
unispeech_sat [Wav2Vec2 and Co] Update init tests for PT 2.1 (#26494) 2023-10-03 10:52:34 +02:00
upernet Fix last models for common tests that are too big. (#25058) 2023-07-25 07:56:04 -04:00
videomae Input data format (#25464) 2023-08-16 17:45:02 +01:00
vilt [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
vision_encoder_decoder Safetensors serialization by default (#27064) 2023-10-31 19:16:49 +01:00
vision_text_dual_encoder
visual_bert [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
vit device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
vit_hybrid fix vit hybrid test (#25543) 2023-08-16 17:02:57 +02:00
vit_mae CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
vit_msn CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
vitdet Skip offload tests for ViTDet (#25913) 2023-09-04 11:35:39 +02:00
vitmatte [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
vits [Tokenizers] Skip tests temporarily (#26574) 2023-10-03 19:43:42 +02:00
vivit Input data format (#25464) 2023-08-16 17:45:02 +01:00
wav2vec2 device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
wav2vec2_conformer device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
wav2vec2_phoneme
wav2vec2_with_lm Fix some tests using "common_voice" (#27147) 2023-10-30 15:27:15 +01:00
wavlm [Wav2Vec2 and Co] Update init tests for PT 2.1 (#26494) 2023-10-03 10:52:34 +02:00
whisper device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
x_clip [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
xglm device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
xlm CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
xlm_prophetnet
xlm_roberta
xlm_roberta_xl CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
xlnet 🚨🚨 🚨🚨 [Tokenizer] attemp to fix add_token issues🚨🚨 🚨🚨 (#23909) 2023-09-18 20:28:36 +02:00
xmod CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
yolos Refactor image processor testers (#25450) 2023-08-11 11:30:18 +01:00
yoso CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
__init__.py