mirror of
https://github.com/huggingface/transformers.git
synced 2025-07-13 09:40:06 +06:00
![]() * Add WhisperTokenizerFast * Fixup * Up * Up * Improve tests * Update src/transformers/models/whisper/tokenization_whisper_fast.py Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com> * Keep stride in whisper pipelien test * Remove unknown token special case * Reduce vocabulary size in tests * Fix vocab size assertion * Sync copied changes from WhisperTokenizer * Skip pipeline tests * Update assertion * Remove Whisper tokenizer dependency on sentencepiece * Format --------- Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com> |
||
---|---|---|
.. | ||
de | ||
en | ||
es | ||
fr | ||
it | ||
ja | ||
ko | ||
pt | ||
zh | ||
_config.py |