mirror of
https://github.com/saymrwulf/transformers.git
synced 2026-05-14 20:58:08 +00:00
832c6191ed
* update modular_modernbert -- add inputs_embeds param to ModernBertModel * Fix implementation issues; extend to other classes; docstring First of all, the inputs_embeds shouldn't fully replace `self.embeddings(input_ids)`, because this call also does layer normalization and dropout. So, now both input_ids and inputs_embeds is passed to the ModernBertEmbeddings, much like how BertEmbeddings is implemented. I also added `inputs_embeds` to the docstring, and propagated the changes to the other model classes. I also introduced an error if input_ids and input_embeds are both or neither provided. Lastly, I fixed an issue with device being based solely on input_ids with attention_mask. * Propagate inputs_embeds to ModernBertForMaskedLM correctly Also reintroduce inputs_embeds test --------- Co-authored-by: Tom Aarsen <Cubiegamedev@gmail.com> |
||
|---|---|---|
| .. | ||
| agents | ||
| benchmark | ||
| bettertransformer | ||
| deepspeed | ||
| extended | ||
| fixtures | ||
| fsdp | ||
| generation | ||
| models | ||
| optimization | ||
| peft_integration | ||
| pipelines | ||
| quantization | ||
| repo_utils | ||
| sagemaker | ||
| tokenization | ||
| tp | ||
| trainer | ||
| utils | ||
| __init__.py | ||
| test_backbone_common.py | ||
| test_configuration_common.py | ||
| test_feature_extraction_common.py | ||
| test_image_processing_common.py | ||
| test_image_transforms.py | ||
| test_modeling_common.py | ||
| test_modeling_flax_common.py | ||
| test_modeling_tf_common.py | ||
| test_pipeline_mixin.py | ||
| test_processing_common.py | ||
| test_sequence_feature_extraction_common.py | ||
| test_tokenization_common.py | ||