Raushan Turganbay 187439c3fa VLM: special multimodal Tokenizer (#34461) ... * kinda works * update * add tests * update * use special tokens in processors * typo * fix copies * fix * fix moshi after rebase * update * fix tests * update * Update docs/source/en/main_classes/tokenizer.md Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com> * update docs * test for load time adding tokens * fix some more tests which are now fetched better * one more fix --------- Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>		2024-11-04 16:37:51 +01:00
..
__init__.py	Moshi integration (#33624)	2024-10-16 11:21:49 +02:00
test_modeling_moshi.py	Tests: move generate tests to the right mixin and delete redundant tests (#34464)	2024-10-30 10:59:08 +00:00
test_tokenization_moshi.py	VLM: special multimodal Tokenizer (#34461)	2024-11-04 16:37:51 +01:00