riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-08-01 02:31:11 +06:00
Code Issues Actions 3 Packages Projects Releases Wiki Activity

add max_length to showcase the use of truncation (#8975)

Browse Source
This commit is contained in:
Navjot 2020-12-07 15:35:39 -08:00 committed by GitHub
parent 62d30e0583
commit c108d0b5a4
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 2 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/source/quicktour.rst
View File

@ -182,6 +182,7 @@ and get tensors back. You can specify all of that to the tokenizer:
... ["We are very happy to show you the 🤗 Transformers library.", "We hope you don't hate it."],
... padding=True,
... truncation=True,
... max_length=512,
... return_tensors="pt"
... )
>>> ## TENSORFLOW CODE
@ -189,6 +190,7 @@ and get tensors back. You can specify all of that to the tokenizer:
... ["We are very happy to show you the 🤗 Transformers library.", "We hope you don't hate it."],
... padding=True,
... truncation=True,
... max_length=512,
... return_tensors="tf"
... )