mirror of
https://github.com/huggingface/transformers.git
synced 2025-07-14 10:08:29 +06:00
f3065abdb8
* Start doc tokenizers * Tokenizer documentation * Start doc tokenizers * Tokenizer documentation * Formatting after rebase * Formatting after merge * Update docs/source/main_classes/tokenizer.rst Co-authored-by: Lysandre Debut <lysandre@huggingface.co> * Address comment * Update src/transformers/tokenization_utils_base.py Co-authored-by: Thomas Wolf <thomwolf@users.noreply.github.com> * Address Thom's comments Co-authored-by: Lysandre Debut <lysandre@huggingface.co> Co-authored-by: Thomas Wolf <thomwolf@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| configuration.rst | ||
| model.rst | ||
| optimizer_schedules.rst | ||
| output.rst | ||
| pipelines.rst | ||
| processors.rst | ||
| tokenizer.rst | ||
| trainer.rst | ||