riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-08-02 19:21:31 +06:00
Code Issues Actions 3 Packages Projects Releases Wiki Activity
f93c1e9ece
transformers/tests/models/transfo_xl
History
Joel Tang dbf7bfafa7
Fix idx2sym not loaded from pretrained vocab file in Transformer XL (#27589) 
* Load idx2sym from pretrained vocab file in Transformer XL

When loading vocab file from a pretrained tokenizer for Transformer XL,
although the pickled vocabulary file contains a idx2sym key, it isn't
loaded, because it is discarded as the empty list already exists as
an attribute.

Solution is to explicitly take it into account, just like for sym2idx.

* ran make style
2023-11-20 07:56:18 +01:00
..
__init__.py Move test model folders (#17034) 2022-05-03 14:42:02 +02:00
test_modeling_tf_transfo_xl.py [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
test_modeling_transfo_xl.py [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
test_tokenization_transfo_xl.py Fix idx2sym not loaded from pretrained vocab file in Transformer XL (#27589) 2023-11-20 07:56:18 +01:00