Raushan Turganbay 187439c3fa VLM: special multimodal Tokenizer (#34461) ... * kinda works * update * add tests * update * use special tokens in processors * typo * fix copies * fix * fix moshi after rebase * update * fix tests * update * Update docs/source/en/main_classes/tokenizer.md Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com> * update docs * test for load time adding tokens * fix some more tests which are now fetched better * one more fix --------- Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>		2024-11-04 16:37:51 +01:00
..
__init__.py	LLaMA Implementation (#21955)	2023-03-16 09:00:53 -04:00
test_modeling_flax_llama.py	Add Llama Flax Implementation (#24587)	2023-12-07 07:05:00 +01:00
test_modeling_llama.py	avoid calling gc.collect and cuda.empty_cache (#34514)	2024-10-31 16:36:13 +01:00
test_tokenization_llama.py	VLM: special multimodal Tokenizer (#34461)	2024-11-04 16:37:51 +01:00