riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-31 02:02:21 +06:00
Code Issues Actions 33 Packages Projects Releases Wiki Activity
deafc24388
transformers/tests/pipelines
History
Jonatan Kłosko deafc24388
Add WhisperTokenizerFast (#21222) 
* Add WhisperTokenizerFast

* Fixup

* Up

* Up

* Improve tests

* Update src/transformers/models/whisper/tokenization_whisper_fast.py

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* Keep stride in whisper pipelien test

* Remove unknown token special case

* Reduce vocabulary size in tests

* Fix vocab size assertion

* Sync copied changes from WhisperTokenizer

* Skip pipeline tests

* Update assertion

* Remove Whisper tokenizer dependency on sentencepiece

* Format

---------

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
2023-02-21 06:58:54 +01:00
..
__init__.py [Test refactor 1/5] Per-folder tests reorganization (#15725) 2022-02-23 15:46:28 -05:00
test_pipelines_audio_classification.py Pipeline testing - using tiny models on Hub (#20426) 2023-01-30 10:39:43 +01:00
test_pipelines_automatic_speech_recognition.py Add WhisperTokenizerFast (#21222) 2023-02-21 06:58:54 +01:00
test_pipelines_common.py refactor: Make direct_transformers_import util (#21652) 2023-02-16 11:32:32 -05:00
test_pipelines_conversational.py Pipeline testing - using tiny models on Hub (#20426) 2023-01-30 10:39:43 +01:00
test_pipelines_depth_estimation.py Update quality tooling for formatting (#21480) 2023-02-06 18:10:56 -05:00
test_pipelines_document_question_answering.py Fix some pipeline tests (#21401) 2023-02-02 19:03:31 +01:00
test_pipelines_feature_extraction.py Pipeline testing - using tiny models on Hub (#20426) 2023-01-30 10:39:43 +01:00
test_pipelines_fill_mask.py Pipeline testing - using tiny models on Hub (#20426) 2023-01-30 10:39:43 +01:00
test_pipelines_image_classification.py Fix some pipeline tests (#21401) 2023-02-02 19:03:31 +01:00
test_pipelines_image_segmentation.py Update quality tooling for formatting (#21480) 2023-02-06 18:10:56 -05:00
test_pipelines_image_to_text.py Fix some pipeline tests (#21401) 2023-02-02 19:03:31 +01:00
test_pipelines_object_detection.py Fix some pipeline tests (#21401) 2023-02-02 19:03:31 +01:00
test_pipelines_question_answering.py Pipeline testing - using tiny models on Hub (#20426) 2023-01-30 10:39:43 +01:00
test_pipelines_summarization.py Pipeline testing - using tiny models on Hub (#20426) 2023-01-30 10:39:43 +01:00
test_pipelines_table_question_answering.py Add require_torch to 2 pipeline tests (#20585) 2022-12-05 16:06:39 +01:00
test_pipelines_text_classification.py Pipeline testing - using tiny models on Hub (#20426) 2023-01-30 10:39:43 +01:00
test_pipelines_text_generation.py [pipeline] A simple fix for half-precision & 8bit models (#21479) 2023-02-10 10:26:17 +01:00
test_pipelines_text2text_generation.py Pipeline testing - using tiny models on Hub (#20426) 2023-01-30 10:39:43 +01:00
test_pipelines_token_classification.py Pipeline testing - using tiny models on Hub (#20426) 2023-01-30 10:39:43 +01:00
test_pipelines_translation.py Pipeline testing - using tiny models on Hub (#20426) 2023-01-30 10:39:43 +01:00
test_pipelines_video_classification.py Update quality tooling for formatting (#21480) 2023-02-06 18:10:56 -05:00
test_pipelines_visual_question_answering.py Pipeline testing - using tiny models on Hub (#20426) 2023-01-30 10:39:43 +01:00
test_pipelines_zero_shot_image_classification.py Pipeline testing - using tiny models on Hub (#20426) 2023-01-30 10:39:43 +01:00
test_pipelines_zero_shot_object_detection.py Update quality tooling for formatting (#21480) 2023-02-06 18:10:56 -05:00
test_pipelines_zero_shot.py Pipeline testing - using tiny models on Hub (#20426) 2023-01-30 10:39:43 +01:00