riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-08-02 03:01:07 +06:00
Code Issues Actions 3 Packages Projects Releases Wiki Activity

better for python2.x

Browse Source
This commit is contained in:
Pascal Voitot 2019-12-10 09:58:06 +01:00 committed by Lysandre Debut
parent 4cbdc7d910
commit f2ac50cb55
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
transformers/tests/tokenization_bert_test.py
View File

@ -109,7 +109,7 @@ class BertTokenizationTest(CommonTestCases.CommonTokenizerTester):
decoded = tokenizer.decode(encoded)
self.assertEqual(
decoded.lower(),
(f"[CLS] {input} [SEP]").lower()
("[CLS] " + input + " [SEP]").lower()
)
def test_is_whitespace(self):