saymrwulf/transformers
1
0
Fork 0
mirror of https://github.com/saymrwulf/transformers.git synced 2026-05-14 20:58:08 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Generation: past kv can be None (#31051)

Browse source 
* fix

* better
This commit is contained in:
Raushan Turganbay 2024-06-27 09:55:33 +05:00 committed by GitHub
parent 1de7dc7403
commit dc76e9fa7f
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
src/transformers/generation/utils.py
View file

@ -1394,7 +1394,7 @@ class GenerationMixin:
return model_kwargs
past_length = 0
if "past_key_values" in model_kwargs:
if model_kwargs.get("past_key_values") is not None:
if isinstance(model_kwargs["past_key_values"], Cache):
past_length = model_kwargs["past_key_values"].get_seq_length()
else: