transformers/docs/source/main_classes/tokenizer.rst
2019-08-04 12:14:57 +02:00

9 lines
184 B
ReStructuredText

Tokenizer
----------------------------------------------------
``PreTrainedTokenizer``
~~~~~~~~~~~~~~~~~~~~~~~~
.. autoclass:: pytorch_transformers.PreTrainedTokenizer
:members: