riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-08-01 18:51:14 +06:00
Code Issues Actions 3 Packages Projects Releases Wiki Activity

Generation: past kv can be None (#31051)

Browse Source 
* fix

* better
This commit is contained in:
Raushan Turganbay 2024-06-27 09:55:33 +05:00 committed by GitHub
parent 1de7dc7403
commit dc76e9fa7f
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
src/transformers/generation/utils.py
View File

@ -1394,7 +1394,7 @@ class GenerationMixin:
return model_kwargs return model_kwargs
past_length = 0 past_length = 0
if "past_key_values" in model_kwargs: if model_kwargs.get("past_key_values") is not None:
if isinstance(model_kwargs["past_key_values"], Cache): if isinstance(model_kwargs["past_key_values"], Cache):
past_length = model_kwargs["past_key_values"].get_seq_length() past_length = model_kwargs["past_key_values"].get_seq_length()
else: else: