riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-05 13:50:13 +06:00
Code Issues Actions 26 Packages Projects Releases Wiki Activity
10e8cd0d63
transformers/tests/tokenization
History
Arthur eb4579cf43
tokenizer train from iterator without pre_tokenizers (#35396) 
* fix if else issues

* add a test

* fix the test

* style
2025-01-09 15:34:43 +01:00
..
__init__.py [Test refactor 1/5] Per-folder tests reorganization (#15725) 2022-02-23 15:46:28 -05:00
test_tokenization_fast.py Skip tests properly (#31308) 2024-06-26 21:59:08 +01:00
test_tokenization_utils.py tokenizer train from iterator without pre_tokenizers (#35396) 2025-01-09 15:34:43 +01:00