Alex Brooks 71cc8161b2 Granite Vision Support (#35579) ... * Add multimodal granite support Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com> Support multiple image feature layres Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com> * Remove failing validation for visual encoders with no cls Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com> * Update llava based models / configs to support list of feature layers Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com> * Add tests for multiple feature layers Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com> * Use conditional instead of except for misaligned feature shapes Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com> * crop cls from each hidden state Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com> * Fix formatting Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com> * Support single vision feature int in vipllava Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com> * Fix typo in vision feature selection strategy validation Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com> * Add tentative integration test for granite vision models Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com> * Add granite vision docs Replace multimodal granite refs with granite vision Add granite vision / llava next alias Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com> * Use image url in granitevision example Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com> --------- Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com> Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com>		2025-01-23 17:15:52 +01:00
..
agents	Change is_soundfile_availble to is_soundfile_available (#35030)	2025-01-03 14:37:42 +01:00
bettertransformer
deepspeed	Use inherit tempdir makers for tests + fix failing DS tests (#35600)	2025-01-10 10:01:58 -05:00
extended	[tests] skip tests for xpu (#33553)	2024-09-19 19:28:04 +01:00
fixtures
fsdp	[tests] make cuda-only tests device-agnostic (#35607)	2025-01-13 14:48:39 +01:00
generation	[Backend support] Allow num_logits_to_keep as Tensor + add flag (#35757)	2025-01-23 09:47:54 +01:00
models	Granite Vision Support (#35579)	2025-01-23 17:15:52 +01:00
optimization
peft_integration	added logic for deleting adapters once loaded (#34650)	2025-01-06 18:36:40 +00:00
pipelines	fix low-precision audio classification pipeline (#35435)	2025-01-20 16:20:51 +00:00
quantization	Fix : Nemotron tokenizer for GGUF format (#35836)	2025-01-22 12:28:40 +01:00
repo_utils	Fix modular edge case + modular sorting order (#35562)	2025-01-09 17:17:52 +01:00
sagemaker	Trainer - deprecate tokenizer for processing_class (#32385)	2024-10-02 14:08:46 +01:00
tokenization	tokenizer train from iterator without pre_tokenizers (#35396)	2025-01-09 15:34:43 +01:00
tp	Simplify Tensor Parallel implementation with PyTorch TP (#34184)	2024-11-18 19:51:49 +01:00
trainer	Support adamw_torch_8bit (#34993)	2025-01-21 14:17:49 +01:00
utils	[Backend support] Allow num_logits_to_keep as Tensor + add flag (#35757)	2025-01-23 09:47:54 +01:00
__init__.py
test_backbone_common.py
test_configuration_common.py	Load sub-configs from composite configs (#34410)	2024-11-05 11:34:01 +01:00
test_feature_extraction_common.py
test_image_processing_common.py	add Qwen2-VL image processor fast (#35733)	2025-01-21 11:49:05 -05:00
test_image_transforms.py
test_modeling_common.py	[Backend support] Allow num_logits_to_keep as Tensor + add flag (#35757)	2025-01-23 09:47:54 +01:00
test_modeling_flax_common.py	🚨All attention refactor🚨 (#35235)	2024-12-18 16:53:39 +01:00
test_modeling_tf_common.py	🚨All attention refactor🚨 (#35235)	2024-12-18 16:53:39 +01:00
test_pipeline_mixin.py	Add image text to text pipeline (#34170)	2024-10-31 15:48:11 -04:00
test_processing_common.py	VLMs: major clean up 🧼 (#34502)	2025-01-08 10:35:23 +01:00
test_sequence_feature_extraction_common.py
test_tokenization_common.py	[tokenizers] Ensure that add_prefix_space is propagated to backend_tokenizer.pre_tokenizer (#35593)	2025-01-09 17:46:50 +01:00