Arie Pratama Sutiono f2bf4fcf3d Add SplinterTokenizer unit test (#32652) ... * add unit tests for splinter_tokenizer * add unit test for splinter tokenizer, pass in the question_token to be saved on save_pretrained called * remove unused import * remove vocab_splinter.txt, add Copied from, use fmt:on and fmt:off to prevent autoformatting on long lines * remove all the spaces Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com> --------- Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>		2024-10-03 14:49:56 +02:00
..
__init__.py	Move test model folders (#17034)	2022-05-03 14:42:02 +02:00
test_modeling_splinter.py	update ruff version (#30932)	2024-05-22 06:40:15 +02:00
test_tokenization_splinter.py	Add SplinterTokenizer unit test (#32652)	2024-10-03 14:49:56 +02:00