mirror of
https://github.com/huggingface/transformers.git
synced 2025-07-24 23:08:57 +06:00
9 lines
184 B
ReStructuredText
9 lines
184 B
ReStructuredText
Tokenizer
|
|
----------------------------------------------------
|
|
|
|
``PreTrainedTokenizer``
|
|
~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
.. autoclass:: pytorch_transformers.PreTrainedTokenizer
|
|
:members:
|