saymrwulf/transformers
1
0
Fork 0
mirror of https://github.com/saymrwulf/transformers.git synced 2026-05-14 20:58:08 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

[Moshi] disable automatic compilation if the model can't compile (#35992)

Browse source 
moshi cant compile
This commit is contained in:
Joao Gante 2025-01-31 15:53:06 +00:00 committed by GitHub
parent e6f4a4ebbf
commit 5bbee12ac9
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
src/transformers/generation/utils.py
View file

@ -3183,7 +3183,7 @@ class GenerationMixin:
model_forward = self.__call__
if isinstance(model_kwargs.get("past_key_values"), Cache):
is_compileable = model_kwargs["past_key_values"].is_compileable
is_compileable = model_kwargs["past_key_values"].is_compileable and self._supports_static_cache
if is_compileable and (
self.device.type == "cuda" or generation_config.compile_config._compile_all_devices
):