riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-04 05:10:06 +06:00
Code Issues Actions 20 Packages Projects Releases Wiki Activity

Fix oubli

Browse Source
This commit is contained in:
VictorSanh 2018-11-01 15:44:52 -04:00
parent e1bb7904d8
commit 7af7f8173b
1 changed files with 4 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
tokenization_test_pytorch.py
View File

@ -119,3 +119,7 @@ class TokenizationTest(unittest.TestCase):
self.assertFalse(tokenization._is_punctuation(u"A"))
self.assertFalse(tokenization._is_punctuation(u" "))
if __name__ == '__main__':
unittest.main()