riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-08-02 03:01:07 +06:00
Code Issues Actions 3 Packages Projects Releases Wiki Activity

typo

Browse Source
This commit is contained in:
Yongbo Wang 2019-02-20 21:10:07 +08:00 committed by GitHub
parent 8337740754
commit 813e4d18ba
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
pytorch_pretrained_bert/tokenization.py
View File

@ -63,7 +63,7 @@ def load_vocab(vocab_file):
def whitespace_tokenize(text): def whitespace_tokenize(text):
"""Runs basic whitespace cleaning and splitting on a peice of text.""" """Runs basic whitespace cleaning and splitting on a piece of text."""
text = text.strip() text = text.strip()
if not text: if not text:
return [] return []