Sylvain Gugger f3065abdb8 Doc tokenizer (#6110) ... * Start doc tokenizers * Tokenizer documentation * Start doc tokenizers * Tokenizer documentation * Formatting after rebase * Formatting after merge * Update docs/source/main_classes/tokenizer.rst Co-authored-by: Lysandre Debut <lysandre@huggingface.co> * Address comment * Update src/transformers/tokenization_utils_base.py Co-authored-by: Thomas Wolf <thomwolf@users.noreply.github.com> * Address Thom's comments Co-authored-by: Lysandre Debut <lysandre@huggingface.co> Co-authored-by: Thomas Wolf <thomwolf@users.noreply.github.com>		2020-07-30 14:51:19 -04:00
..
modeling_utils.rst	Model utils doc (#6005)	2020-07-24 09:16:28 -04:00
tokenization_utils.rst	Doc tokenizer (#6110)	2020-07-30 14:51:19 -04:00