riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-31 02:02:21 +06:00
Code Issues Actions 23 Packages Projects Releases Wiki Activity
623d8cb475
transformers/tests
History
Nicolas Patry 623d8cb475
Adding support for microphone streaming within pipeline. (#15046) 
* Adding support for `microphone` streaming within pipeline.

- Uses `ffmpeg` to get microphone data.
- Makes sure alignment is made to `size_of_sample`.
- Works by sending `{"raw": ..data.., "stride": (n, left, right),
"partial": bool}`
directly to the pipeline enabling to stream partial results and still
get inference.
- Let's `partial` information flow through the pipeline to enable caller
  to get it back and choose to display text or not.

- The striding reconstitution is bound to have errors since CTC does not
keep previous state. Currently most of the errors are we don't know if
there's a space or not between two chunks.
Since we have some left striding info, we could use that during decoding
to choose what to do with those spaces and even extra letters maybe (if
the stride is long enough, it's bound to cover at least a few symbols)

Fixing tests.

Protecting with `require_torch`.

`raw_ctc` support for nicer demo.

Post rebase fixes.

Revamp to split raw_mic_data from it's live chunking.

- Requires a refactor to make everything a bit cleaner.

Automatic resampling.

Small fix.

Small fix.

* Post rebase fix (need to let super handle more logic, reorder args.)

* Update docstrings

* Docstring format.

* Remove print.

* Prevent flow of `input_values`.

* Fixing `stride` too.

* Fixing the PR by removing `raw_ctc`.

* Better docstrings.

* Fixing init.

* Update src/transformers/pipelines/audio_utils.py

Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>

* Update tests/test_pipelines_automatic_speech_recognition.py

Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>

* Quality.

Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
2022-02-02 15:12:12 +01:00
..
deepspeed [deepspeed tests] fix summarization (#15149) 2022-01-13 13:48:51 -08:00
extended Fix failing GPU trainer tests (#14903) 2021-12-23 13:59:33 -05:00
fixtures Add model like (#14992) 2022-01-24 15:25:10 -05:00
sagemaker SageMaker: Fix sagemaker DDP & metric logs (#13181) 2021-08-23 10:18:07 +02:00
__init__.py GPU text generation: mMoved the encoded_prompt to correct device 2020-01-06 15:11:12 +01:00
conftest.py Only test the files impacted by changes in the diff (#12644) 2021-07-14 10:56:55 -04:00
test_activations_tf.py add TF glu activation function (#15146) 2022-01-14 10:42:08 +00:00
test_activations.py Fix SEW-D implementation differences (#14191) 2021-10-28 16:22:18 +03:00
test_add_new_model_like.py Add model like (#14992) 2022-01-24 15:25:10 -05:00
test_benchmark_tf.py Fix from_pretrained with corrupted state_dict (#12939) 2021-08-04 11:48:39 +02:00
test_benchmark.py Fix from_pretrained with corrupted state_dict (#12939) 2021-08-04 11:48:39 +02:00
test_cli.py Replace assert with unittest assertions (#13957) 2021-10-11 10:21:46 -04:00
test_configuration_auto.py Allow relative imports in dynamic code (#15352) 2022-01-27 14:47:59 -05:00
test_configuration_common.py Avoid using get_list_of_files (#15287) 2022-01-25 09:41:21 -05:00
test_data_collator.py Fixes tf_default_data_collator sometimes guessing the wrong dtype for labels (#15234) 2022-01-20 14:26:51 +00:00
test_doc_samples.py [doc] :obj: hunt (#14954) 2021-12-27 15:49:48 -08:00
test_feature_extraction_auto.py Refine errors for pretrained objects (#15261) 2022-01-21 15:00:09 -05:00
test_feature_extraction_beit.py Improve semantic segmentation models (#14355) 2021-11-17 15:29:58 +01:00
test_feature_extraction_clip.py CLIP (#11445) 2021-05-12 13:48:15 +05:30
test_feature_extraction_common.py Add DETR (#11653) 2021-06-09 11:51:13 -04:00
test_feature_extraction_deit.py Add DETR (#11653) 2021-06-09 11:51:13 -04:00
test_feature_extraction_detr.py Rename detr targets to labels (#12280) 2021-06-29 03:07:46 -04:00
test_feature_extraction_imagegpt.py Add ImageGPT (#14240) 2021-11-18 16:24:34 +01:00
test_feature_extraction_layoutlmv2.py Add LayoutLMv2 + LayoutXLM (#12604) 2021-08-30 12:35:42 +02:00
test_feature_extraction_segformer.py Improve semantic segmentation models (#14355) 2021-11-17 15:29:58 +01:00
test_feature_extraction_speech_to_text.py [Wav2Vec2FeatureExtractor] Fix extractor.pad() dtype backwards compatibility (#13693) 2021-09-22 11:02:54 +02:00
test_feature_extraction_vilt.py Add ViLT (#14895) 2022-01-19 19:51:59 +01:00
test_feature_extraction_vit.py Add DETR (#11653) 2021-06-09 11:51:13 -04:00
test_feature_extraction_wav2vec2.py [Wav2Vec2FeatureExtractor] Fix extractor.pad() dtype backwards compatibility (#13693) 2021-09-22 11:02:54 +02:00
test_file_utils.py Avoid using get_list_of_files (#15287) 2022-01-25 09:41:21 -05:00
test_generation_beam_search.py Remove max length beam scorer (#11378) 2021-04-27 00:28:40 +02:00
test_generation_flax_logits_process.py [Flax] Add Beam Search (#12131) 2021-06-16 09:43:54 +01:00
test_generation_flax_utils.py [Flax Generation] Correct inconsistencies PyTorch/Flax (#12662) 2021-07-13 18:53:30 +01:00
test_generation_logits_process.py [style] consistent nn. and nn.functional: part 3 tests (#12155) 2021-06-14 12:18:22 -07:00
test_generation_stopping_criteria.py Adding new argument max_new_tokens for generate. (#11476) 2021-05-27 14:22:58 +02:00
test_generation_utils.py [Beam Search] Correct returned beam scores (#14654) 2022-01-24 21:13:21 +01:00
test_hf_argparser.py Update no_* argument (HfArgumentParser) (#13865) 2021-10-04 16:28:52 -04:00
test_image_utils.py Add option to resize like torchvision's Resize (#15419) 2022-02-02 09:44:22 +01:00
test_logging.py Enable tqdm toggling (#15167) 2022-01-18 17:52:35 -05:00
test_model_card.py GPU text generation: mMoved the encoded_prompt to correct device 2020-01-06 15:11:12 +01:00
test_model_output.py Fix ModelOutput instantiation form dictionaries (#13067) 2021-08-10 12:20:04 +02:00
test_modeling_albert.py Add 'with torch.no_grad()' to integration test forward pass (#14808) 2021-12-23 04:23:39 -05:00
test_modeling_auto.py Allow relative imports in dynamic code (#15352) 2022-01-27 14:47:59 -05:00
test_modeling_bart.py Moving question_answering tests to the new testing scheme. Had to tweak a little some ModelTesterConfig for pipelines. (#13277) 2021-08-26 12:37:55 +02:00
test_modeling_beit.py Add 'with torch.no_grad()' to BEiT integration test forward passes (#14961) 2022-01-31 15:12:10 -05:00
test_modeling_bert_generation.py Add 'with torch.no_grad()' to BertGeneration integration test forward passes (#14963) 2022-01-06 10:39:13 -05:00
test_modeling_bert.py Add 'with torch.no_grad()' to integration test forward pass (#14820) 2021-12-20 09:28:17 -05:00
test_modeling_big_bird.py Generalize problem_type to all sequence classification models (#14180) 2021-10-29 10:32:56 -04:00
test_modeling_bigbird_pegasus.py Expose get_config() on ModelTesters (#12812) 2021-07-21 04:13:11 -04:00
test_modeling_blenderbot_small.py Expose get_config() on ModelTesters (#12812) 2021-07-21 04:13:11 -04:00
test_modeling_blenderbot.py Add BlenderbotTokenizerFast (#13720) 2021-10-29 09:19:01 -04:00
test_modeling_bort.py ADD BORT (#9813) 2021-01-27 21:25:11 +03:00
test_modeling_camembert.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_modeling_canine.py Fix ignore_mismatched_sizes (#14085) 2021-10-21 12:31:29 -04:00
test_modeling_clip.py [CLIP] Fix PT test (#15041) 2022-01-05 14:21:04 +01:00
test_modeling_common.py Add a main_input_name attribute to all models (#14803) 2021-12-20 11:19:08 -05:00
test_modeling_convbert.py Generalize problem_type to all sequence classification models (#14180) 2021-10-29 10:32:56 -04:00
test_modeling_ctrl.py Remove n_ctx from configs (#14165) 2021-10-29 11:50:25 +02:00
test_modeling_deberta_v2.py Expose get_config() on ModelTesters (#12812) 2021-07-21 04:13:11 -04:00
test_modeling_deberta.py Add 'with torch.no_grad()' to integration test forward pass (#14821) 2021-12-20 09:25:34 -05:00
test_modeling_deit.py [Tests] Improve vision tests (#14458) 2021-11-24 15:22:20 +01:00
test_modeling_detr.py Expose get_config() on ModelTesters (#12812) 2021-07-21 04:13:11 -04:00
test_modeling_distilbert.py Add with torch.no_grad() to DistilBERT integration test forward pass (#14979) 2022-01-12 10:42:39 -05:00
test_modeling_dpr.py [DPR] Correct init (#13796) 2021-09-30 18:55:20 +02:00
test_modeling_electra.py Add ElectraForCausalLM -> Enable Electra encoder-decoder model (#14729) 2021-12-27 12:37:52 +01:00
test_modeling_encoder_decoder.py Fix usage of additional kwargs in from_encoder_decoder_pretrained in encoder-decoder models (#15056) 2022-01-19 23:00:33 +01:00
test_modeling_flaubert.py Fix torchscript tests (#13701) 2021-09-22 19:02:54 -04:00
test_modeling_flax_albert.py albert flax (#13294) 2021-08-30 17:29:27 +02:00
test_modeling_flax_auto.py Refine errors for pretrained objects (#15261) 2022-01-21 15:00:09 -05:00
test_modeling_flax_bart.py [Flax] Correct all return tensors to numpy (#13307) 2021-08-27 17:38:34 +02:00
test_modeling_flax_beit.py finish (#13743) 2021-09-25 21:20:21 +02:00
test_modeling_flax_bert.py Implement head_mask for Flax BERT and other models copied from BERT (#14620) 2021-12-17 17:06:59 +01:00
test_modeling_flax_big_bird.py Fix ignore_mismatched_sizes (#14085) 2021-10-21 12:31:29 -04:00
test_modeling_flax_blenderbot_small.py [Flax] Add FlaxBlenderbotSmall (#14576) 2021-12-02 14:21:48 +05:30
test_modeling_flax_blenderbot.py [Flax] Add FlaxBlenderbot (#13633) 2021-11-30 17:36:54 +05:30
test_modeling_flax_clip.py fix loading flax bf16 weights in pt (#14369) 2021-11-11 21:20:49 +05:30
test_modeling_flax_common.py Add a main_input_name attribute to all models (#14803) 2021-12-20 11:19:08 -05:00
test_modeling_flax_distilbert.py distilbert-flax (#13324) 2021-08-30 14:16:18 +02:00
test_modeling_flax_electra.py Implement head_mask for Flax BERT and other models copied from BERT (#14620) 2021-12-17 17:06:59 +01:00
test_modeling_flax_encoder_decoder.py Use cross_attention_hidden_size in Encoder-Decoder models (#14378) 2021-12-07 00:27:32 +01:00
test_modeling_flax_gpt_neo.py Make gradient_checkpointing a training argument (#13657) 2021-09-22 07:51:38 -04:00
test_modeling_flax_gpt2.py Remove n_ctx from configs (#14165) 2021-10-29 11:50:25 +02:00
test_modeling_flax_gptj.py FlaxGPTJ (#14396) 2021-12-01 10:57:39 +05:30
test_modeling_flax_marian.py [Flax Generation] Correct inconsistencies PyTorch/Flax (#12662) 2021-07-13 18:53:30 +01:00
test_modeling_flax_mbart.py [Flax] Add FlaxMBart (#12236) 2021-07-07 12:20:38 +05:30
test_modeling_flax_mt5.py Add require flax to test (#13260) 2021-08-25 12:56:25 -04:00
test_modeling_flax_pegasus.py [Flax] Addition of FlaxPegasus (#13420) 2021-09-14 17:15:19 +02:00
test_modeling_flax_roberta.py Implement head_mask for Flax BERT and other models copied from BERT (#14620) 2021-12-17 17:06:59 +01:00
test_modeling_flax_roformer.py Add Flax RoFormer (#15005) 2022-01-04 13:23:10 +01:00
test_modeling_flax_t5.py fix loading flax bf16 weights in pt (#14369) 2021-11-11 21:20:49 +05:30
test_modeling_flax_vision_encoder_decoder.py Add FlaxVisionEncoderDecoderModel (#13359) 2021-11-09 15:14:28 +05:30
test_modeling_flax_vision_text_dual_encoder.py VisionTextDualEncoder (#13511) 2021-11-30 22:21:48 +05:30
test_modeling_flax_vit.py [Flax/JAX] Run jitted tests at every commit (#13090) 2021-08-12 14:49:46 +02:00
test_modeling_flax_wav2vec2.py Remove datasets requirement (#14795) 2021-12-16 14:34:14 -05:00
test_modeling_flax_xglm.py Add XGLM models (#14876) 2022-01-28 18:55:23 +01:00
test_modeling_fnet.py Add FNet (#13045) 2021-09-20 13:24:30 +02:00
test_modeling_fsmt.py Expose get_config() on ModelTesters (#12812) 2021-07-21 04:13:11 -04:00
test_modeling_funnel.py Generalize problem_type to all sequence classification models (#14180) 2021-10-29 10:32:56 -04:00
test_modeling_gpt_neo.py Make gradient_checkpointing a training argument (#13657) 2021-09-22 07:51:38 -04:00
test_modeling_gpt2.py Remove n_ctx from configs (#14165) 2021-10-29 11:50:25 +02:00
test_modeling_gptj.py Add GPTJForQuestionAnswering (#14503) 2021-12-06 11:44:10 -05:00
test_modeling_hubert.py [Speech models] Disable non-existing chunking in tests (#15163) 2022-01-16 17:15:19 +01:00
test_modeling_ibert.py Expose get_config() on ModelTesters (#12812) 2021-07-21 04:13:11 -04:00
test_modeling_imagegpt.py [ImageGPT] Deprecate pixel_values input name to input_ids (#14801) 2021-12-17 20:05:22 +01:00
test_modeling_layoutlm.py Expose get_config() on ModelTesters (#12812) 2021-07-21 04:13:11 -04:00
test_modeling_layoutlmv2.py [LayoutLMV2 Tests] Make sure input is on GPU (#15314) 2022-01-24 15:54:47 +01:00
test_modeling_led.py Fix incorrect output shapes for TF/PT LED (#13882) 2021-10-07 17:30:15 +01:00
test_modeling_longformer.py Generalize problem_type to all sequence classification models (#14180) 2021-10-29 10:32:56 -04:00
test_modeling_luke.py Add mLUKE (#14640) 2021-12-07 00:25:28 -05:00
test_modeling_lxmert.py Add TF<>PT and Flax<>PT everywhere (#14047) 2021-10-25 23:55:08 +02:00
test_modeling_m2m_100.py Expose get_config() on ModelTesters (#12812) 2021-07-21 04:13:11 -04:00
test_modeling_marian.py Update Transformers to huggingface_hub >= 0.1.0 (#14251) 2021-11-02 18:58:42 -04:00
test_modeling_mbart.py Improving pipeline tests (#12784) 2021-07-22 15:19:35 +02:00
test_modeling_megatron_bert.py Initial support for symbolic tracing with torch.fx allowing dynamic axes (#13579) 2021-10-05 14:19:47 +02:00
test_modeling_megatron_gpt2.py Skip test while the model is not available (#12740) 2021-07-15 09:14:12 -04:00
test_modeling_mobilebert.py Generalize problem_type to all sequence classification models (#14180) 2021-10-29 10:32:56 -04:00
test_modeling_mpnet.py Expose get_config() on ModelTesters (#12812) 2021-07-21 04:13:11 -04:00
test_modeling_mt5.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_modeling_nystromformer.py Add Nystromformer (#14659) 2022-01-11 14:25:49 +01:00
test_modeling_openai.py Generalize problem_type to all sequence classification models (#14180) 2021-10-29 10:32:56 -04:00
test_modeling_pegasus.py [Pretrained Model] Add resize_position_embeddings (#13559) 2021-09-15 19:03:56 +02:00
test_modeling_perceiver.py [Perceiver] Skip multi-gpu tests for now (#14813) 2021-12-20 15:22:50 +01:00
test_modeling_prophetnet.py Skip ProphetNet test (#12462) 2021-08-04 18:24:54 +02:00
test_modeling_qdqbert.py Add QDQBert model and quantization examples of SQUAD task (#14066) 2021-11-19 13:33:39 -05:00
test_modeling_rag.py [doc] :class: hunt (#14955) 2021-12-27 17:17:38 -08:00
test_modeling_realm.py Change REALM checkpoint to new ones (#15439) 2022-01-31 12:50:20 -05:00
test_modeling_reformer.py [Perceiver] Skip multi-gpu tests for now (#14813) 2021-12-20 15:22:50 +01:00
test_modeling_rembert.py Add RemBERT model code to huggingface (#10692) 2021-07-24 11:31:42 -04:00
test_modeling_roberta.py wrapped forward passes in torch.no_grad() (#15037) 2022-01-06 08:48:49 -05:00
test_modeling_roformer.py Expose get_config() on ModelTesters (#12812) 2021-07-21 04:13:11 -04:00
test_modeling_segformer.py Fix tests (#14289) 2021-11-06 10:08:58 -04:00
test_modeling_sew_d.py [Speech models] Disable non-existing chunking in tests (#15163) 2022-01-16 17:15:19 +01:00
test_modeling_sew.py [Speech models] Disable non-existing chunking in tests (#15163) 2022-01-16 17:15:19 +01:00
test_modeling_speech_encoder_decoder.py Fix scheduled tests for SpeechEncoderDecoderModel (#13422) 2021-09-06 14:55:13 +02:00
test_modeling_speech_to_text_2.py Add SpeechEncoderDecoder & Speech2Text2 (#13186) 2021-09-01 13:33:31 +02:00
test_modeling_speech_to_text.py [Speech] Move all examples to new audio feature (#14045) 2021-10-18 12:52:40 +02:00
test_modeling_splinter.py Add splinter (#12955) 2021-08-17 08:29:01 -04:00
test_modeling_squeezebert.py Generalize problem_type to all sequence classification models (#14180) 2021-10-29 10:32:56 -04:00
test_modeling_swin.py [Hotfix] Fix Swin model outputs (#15414) 2022-01-31 16:32:14 +01:00
test_modeling_t5.py Moving feature-extraction pipeline to new testing scheme (#12843) 2021-07-29 19:35:55 +02:00
test_modeling_tapas.py Expose get_config() on ModelTesters (#12812) 2021-07-21 04:13:11 -04:00
test_modeling_tf_albert.py Add support for multiple models for one config in auto classes (#11150) 2021-04-08 18:41:36 -04:00
test_modeling_tf_auto.py Refine errors for pretrained objects (#15261) 2022-01-21 15:00:09 -05:00
test_modeling_tf_bart.py [WIP] Ensure TF model configs can be converted to proper JSON (#14415) 2021-11-17 20:24:39 +00:00
test_modeling_tf_bert.py [WIP] Ensure TF model configs can be converted to proper JSON (#14415) 2021-11-17 20:24:39 +00:00
test_modeling_tf_blenderbot_small.py TF BART models - Add cross_attentions to model output and fix cross-attention head masking (#10699) 2021-04-26 14:16:21 +02:00
test_modeling_tf_blenderbot.py TF BART models - Add cross_attentions to model output and fix cross-attention head masking (#10699) 2021-04-26 14:16:21 +02:00
test_modeling_tf_bort.py ADD BORT (#9813) 2021-01-27 21:25:11 +03:00
test_modeling_tf_camembert.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_modeling_tf_clip.py [CLIP] Fix TF test (#15042) 2022-01-05 16:58:42 +01:00
test_modeling_tf_common.py remove "inputs" in tf common test script (no longer required) (#15262) 2022-02-01 10:09:49 +00:00
test_modeling_tf_convbert.py ConvBERT fix torch <> tf weights conversion (#10314) 2021-02-24 14:55:34 +03:00
test_modeling_tf_core.py [WIP] Ensure TF model configs can be converted to proper JSON (#14415) 2021-11-17 20:24:39 +00:00
test_modeling_tf_ctrl.py Remove n_ctx from configs (#14165) 2021-10-29 11:50:25 +02:00
test_modeling_tf_deberta_v2.py Deberta_v2 tf (#13120) 2021-08-31 06:32:47 -04:00
test_modeling_tf_deberta.py Deberta tf (#12972) 2021-08-12 05:01:26 -04:00
test_modeling_tf_distilbert.py Check TF ops for ONNX compliance (#10025) 2021-02-15 07:55:10 -05:00
test_modeling_tf_dpr.py Check TF ops for ONNX compliance (#10025) 2021-02-15 07:55:10 -05:00
test_modeling_tf_electra.py Check TF ops for ONNX compliance (#10025) 2021-02-15 07:55:10 -05:00
test_modeling_tf_encoder_decoder.py Fix TF Causal LM models' returned logits (#15256) 2022-02-01 11:04:07 +00:00
test_modeling_tf_flaubert.py Making TF XLM-like models XLA and AMP compliant (#10211) 2021-02-17 18:02:48 +01:00
test_modeling_tf_funnel.py Skip Funnel test 2021-05-05 12:38:01 +02:00
test_modeling_tf_gpt2.py [WIP] Ensure TF model configs can be converted to proper JSON (#14415) 2021-11-17 20:24:39 +00:00
test_modeling_tf_hubert.py [Wav2Vec2] Rename model's feature extractor to feature encoder (#14959) 2021-12-28 20:33:23 +01:00
test_modeling_tf_layoutlm.py remove test_model_various_embeddings (#14341) 2021-11-09 11:30:17 -05:00
test_modeling_tf_led.py Fix TFLEDModel (#15356) 2022-01-31 19:35:54 +01:00
test_modeling_tf_longformer.py Making TF Longformer-like models compliant with AMP (#10233) 2021-02-22 15:41:56 +01:00
test_modeling_tf_lxmert.py implementing tflxmertmodel integration test (#12497) 2021-07-06 11:44:47 -04:00
test_modeling_tf_marian.py Skip TestMarian_MT_EN (#12649) 2021-07-12 09:11:32 -04:00
test_modeling_tf_mbart.py TF BART models - Add cross_attentions to model output and fix cross-attention head masking (#10699) 2021-04-26 14:16:21 +02:00
test_modeling_tf_mobilebert.py Making TF MobileBert model compliant with AMP (#10259) 2021-02-19 06:55:25 -05:00
test_modeling_tf_mpnet.py Making TF MPNet model compliant with XLA (#10260) 2021-02-19 06:56:41 -05:00
test_modeling_tf_mt5.py Adjust loss difference (#10669) 2021-03-12 09:09:46 +03:00
test_modeling_tf_openai.py Remove n_ctx from configs (#14165) 2021-10-29 11:50:25 +02:00
test_modeling_tf_pegasus.py Moving feature-extraction pipeline to new testing scheme (#12843) 2021-07-29 19:35:55 +02:00
test_modeling_tf_pytorch.py Typo "UNKWOWN" -> "UNKNOWN" (#13675) 2021-09-21 09:11:26 -04:00
test_modeling_tf_rag.py [doc] :class: hunt (#14955) 2021-12-27 17:17:38 -08:00
test_modeling_tf_rembert.py Add TFEncoderDecoderModel + Add cross-attention to some TF models (#13222) 2021-10-13 00:10:34 +02:00
test_modeling_tf_roberta.py Add TFEncoderDecoderModel + Add cross-attention to some TF models (#13222) 2021-10-13 00:10:34 +02:00
test_modeling_tf_roformer.py Add new model RoFormer (use rotary position embedding ) (#11684) 2021-05-20 08:00:34 -04:00
test_modeling_tf_t5.py TF Model train and eval step metrics for seq2seq models. (#14009) 2021-10-19 12:14:21 +01:00
test_modeling_tf_tapas.py Tapas tf (#13393) 2021-11-30 11:07:55 +01:00
test_modeling_tf_transfo_xl.py Fix ignore_mismatched_sizes (#14085) 2021-10-21 12:31:29 -04:00
test_modeling_tf_vision_encoder_decoder.py Fix TF Causal LM models' returned logits (#15256) 2022-02-01 11:04:07 +00:00
test_modeling_tf_vit.py Add TFCLIPModel (#13967) 2021-12-23 11:19:44 -05:00
test_modeling_tf_wav2vec2.py [Tests] Correct Wav2Vec2 & WavLM tests (#15015) 2022-01-03 20:19:04 +01:00
test_modeling_tf_xlm_roberta.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_modeling_tf_xlm.py Making TF XLM-like models XLA and AMP compliant (#10211) 2021-02-17 18:02:48 +01:00
test_modeling_tf_xlnet.py Fix the inconsistency of loss calculation between PT/TF XLNetLMHeadModel (#15298) 2022-01-29 15:08:35 +00:00
test_modeling_transfo_xl.py [Perceiver] Skip multi-gpu tests for now (#14813) 2021-12-20 15:22:50 +01:00
test_modeling_trocr.py Add TrOCR + VisionEncoderDecoderModel (#13874) 2021-10-13 10:28:56 +02:00
test_modeling_unispeech_sat.py [Speech models] Disable non-existing chunking in tests (#15163) 2022-01-16 17:15:19 +01:00
test_modeling_unispeech.py [Speech models] Disable non-existing chunking in tests (#15163) 2022-01-16 17:15:19 +01:00
test_modeling_vilt.py [Tests] Fix test (#15324) 2022-01-25 15:48:25 +01:00
test_modeling_vision_encoder_decoder.py [Generate] Remove attention_mask and integrate model_main_input_name (#14856) 2021-12-23 19:43:37 +01:00
test_modeling_vision_text_dual_encoder.py VisionTextDualEncoder (#13511) 2021-11-30 22:21:48 +05:30
test_modeling_visual_bert.py Expose get_config() on ModelTesters (#12812) 2021-07-21 04:13:11 -04:00
test_modeling_vit_mae.py [Tests] Fix test (#15324) 2022-01-25 15:48:25 +01:00
test_modeling_vit.py Added forward pass of test_inference_image_classification_head with torch.no_grad() (#14777) 2022-01-17 07:22:41 -05:00
test_modeling_wav2vec2.py Update modeling_wav2vec2.py (#15423) 2022-01-31 21:22:11 +01:00
test_modeling_wavlm.py [Tests] Correct Wav2Vec2 & WavLM tests (#15015) 2022-01-03 20:19:04 +01:00
test_modeling_xglm.py Add XGLM models (#14876) 2022-01-28 18:55:23 +01:00
test_modeling_xlm_prophetnet.py Ci test tf super slow (#8007) 2020-10-30 10:25:48 -04:00
test_modeling_xlm_roberta_xl.py Add support for XLM-R XL and XXL models by modeling_xlm_roberta_xl.py (#13727) 2022-01-29 13:42:37 +01:00
test_modeling_xlm_roberta.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_modeling_xlm.py Generalize problem_type to all sequence classification models (#14180) 2021-10-29 10:32:56 -04:00
test_modeling_xlnet.py [doc] :obj: hunt (#14954) 2021-12-27 15:49:48 -08:00
test_modeling_yoso.py Add YOSO (#15091) 2022-01-26 19:18:29 +01:00
test_offline.py offline mode for firewalled envs (part 2) (#10569) 2021-03-08 08:52:20 -08:00
test_onnx_v2.py Adds IBERT to models exportable with ONNX (#14868) 2022-01-11 12:17:08 +01:00
test_onnx.py Fixed: Better names for nlp variables in pipelines' tests and docs. (#11752) 2021-05-18 09:47:28 -04:00
test_optimization_tf.py Use stable functions (#9369) 2021-01-05 03:58:26 -05:00
test_optimization.py [style] consistent nn. and nn.functional: part 3 tests (#12155) 2021-06-14 12:18:22 -07:00
test_pipelines_audio_classification.py Remove datasets requirement (#14795) 2021-12-16 14:34:14 -05:00
test_pipelines_automatic_speech_recognition.py Adding support for microphone streaming within pipeline. (#15046) 2022-02-02 15:12:12 +01:00
test_pipelines_common.py Adding QoL for batch_size arg (like others enabled everywhere). (#15027) 2022-01-05 12:16:23 +01:00
test_pipelines_conversational.py Adding batch_size support for (almost) all pipelines (#13724) 2021-10-29 11:34:18 +02:00
test_pipelines_feature_extraction.py Fixing tests on master. (#14317) 2021-11-08 08:28:26 -05:00
test_pipelines_fill_mask.py Adding support for multiple mask tokens. (#14716) 2021-12-14 16:46:16 +01:00
test_pipelines_image_classification.py Enabling TF on image-classification pipeline. (#15030) 2022-01-06 14:16:00 +01:00
test_pipelines_image_segmentation.py Remove datasets requirement (#14795) 2021-12-16 14:34:14 -05:00
test_pipelines_object_detection.py Remove datasets requirement (#14795) 2021-12-16 14:34:14 -05:00
test_pipelines_question_answering.py Handle long answer needs to be updated. (#14279) 2021-11-06 10:04:30 -04:00
test_pipelines_summarization.py Adding batch_size support for (almost) all pipelines (#13724) 2021-10-29 11:34:18 +02:00
test_pipelines_table_question_answering.py Fixing Dataset for TQA + token-classification. (#14658) 2021-12-08 09:54:24 +01:00
test_pipelines_text_classification.py Moving pipeline tests from Narsil to hf-internal-testing. (#14463) 2021-11-22 04:40:45 -05:00
test_pipelines_text_generation.py skip test for XGLM (#15445) 2022-01-31 16:53:16 -05:00
test_pipelines_text2text_generation.py Fixing support batch_size and num_return_Sequences in text-generation pipeline (#15318) 2022-01-28 12:15:30 +01:00
test_pipelines_token_classification.py Fixing Dataset for TQA + token-classification. (#14658) 2021-12-08 09:54:24 +01:00
test_pipelines_translation.py Fixing t2t pipelines lists outputs. (#15008) 2022-01-03 14:49:58 +01:00
test_pipelines_zero_shot.py Adding batch_size support for (almost) all pipelines (#13724) 2021-10-29 11:34:18 +02:00
test_processor_auto.py [AutoProcessor] Correct AutoProcessor and automatically add processor… (#14881) 2021-12-30 09:56:43 +01:00
test_processor_clip.py CLIP (#11445) 2021-05-12 13:48:15 +05:30
test_processor_layoutlmv2.py Add LayoutLMv2 + LayoutXLM (#12604) 2021-08-30 12:35:42 +02:00
test_processor_layoutxlm.py Add LayoutXLMProcessor (and LayoutXLMTokenizer, LayoutXLMTokenizerFast) (#14115) 2021-11-03 08:59:44 +01:00
test_processor_speech_to_text.py Auto feature extractor (#11097) 2021-04-06 19:20:08 -04:00
test_processor_vision_text_dual_encoder.py VisionTextDualEncoder (#13511) 2021-11-30 22:21:48 +05:30
test_processor_wav2vec2_with_lm.py [Wav2Vec2ProcessorWithLM] add alpha & beta to batch decode & decode (#15465) 2022-02-02 12:59:40 +01:00
test_processor_wav2vec2.py Change how "additional_special_tokens" argument in the ".from_pretrained" method of the tokenizer is taken into account (#13056) 2021-08-23 14:35:18 +02:00
test_retrieval_rag.py Remove datasets requirement (#14795) 2021-12-16 14:34:14 -05:00
test_retrieval_realm.py Change REALM checkpoint to new ones (#15439) 2022-01-31 12:50:20 -05:00
test_sequence_feature_extraction_common.py [SequenceFeatureExtractor] Rewrite padding logic from pure python to numpy (#13650) 2021-09-21 17:10:13 +03:00
test_skip_decorators.py [testing] rename skip targets + docs (#7863) 2020-10-20 04:39:13 -04:00
test_tokenization_albert.py Add regression tests for slow sentencepiece tokenizers. (#11737) 2021-06-01 09:24:39 -04:00
test_tokenization_auto.py Refine errors for pretrained objects (#15261) 2022-01-21 15:00:09 -05:00
test_tokenization_bart.py Enforce string-formatting with f-strings (#10980) 2021-03-31 10:00:27 -04:00
test_tokenization_barthez.py Add regression tests for slow sentencepiece tokenizers. (#11737) 2021-06-01 09:24:39 -04:00
test_tokenization_bartpho.py Add BARTpho: Pre-trained Sequence-to-Sequence Models for Vietnamese (#13788) 2021-10-18 10:16:46 -04:00
test_tokenization_bert_generation.py Feature to use the PreTrainedTokenizerFast class as a stand-alone tokenizer (#11810) 2021-06-14 11:58:44 +02:00
test_tokenization_bert_japanese.py Add tokenizers class mismatch detection between cls and checkpoint (#12619) 2021-07-17 15:52:21 +02:00
test_tokenization_bert.py fix BertTokenizerFast tokenize_chinese_chars arg (#15158) 2022-01-14 14:22:03 +01:00
test_tokenization_bertweet.py Feature to use the PreTrainedTokenizerFast class as a stand-alone tokenizer (#11810) 2021-06-14 11:58:44 +02:00
test_tokenization_big_bird.py Patch BigBird tokenization test (#12653) 2021-07-13 02:53:06 -04:00
test_tokenization_blenderbot.py Add BlenderbotTokenizerFast (#13720) 2021-10-29 09:19:01 -04:00
test_tokenization_byt5.py up (#13988) 2021-10-14 10:54:20 +02:00
test_tokenization_camembert.py Add regression tests for slow sentencepiece tokenizers. (#11737) 2021-06-01 09:24:39 -04:00
test_tokenization_canine.py [Tests] Speed up tokenizer tests (#14964) 2021-12-28 17:02:50 +01:00
test_tokenization_clip.py Feature to use the PreTrainedTokenizerFast class as a stand-alone tokenizer (#11810) 2021-06-14 11:58:44 +02:00
test_tokenization_common.py fix the tokenizer_config.json file for the slow tokenizer when a fast version is available (#15319) 2022-02-01 16:48:25 +01:00
test_tokenization_cpm.py Add a special tokenizer for CPM model (#11068) 2021-04-10 02:07:47 +08:00
test_tokenization_ctrl.py Refactor prepare_seq2seq_batch (#9524) 2021-01-12 18:19:38 -05:00
test_tokenization_deberta_v2.py fix deberta 2 tokenizer integration test (#12017) 2021-06-07 04:55:55 -04:00
test_tokenization_deberta.py Implement Fast Tokenization for Deberta (#11387) 2021-04-30 08:08:15 -04:00
test_tokenization_distilbert.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_tokenization_dpr.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_tokenization_fast.py Avoid using get_list_of_files (#15287) 2022-01-25 09:41:21 -05:00
test_tokenization_fnet.py Fix FNet tokenizer tests (#13995) 2021-10-14 09:07:51 -04:00
test_tokenization_fsmt.py Feature to use the PreTrainedTokenizerFast class as a stand-alone tokenizer (#11810) 2021-06-14 11:58:44 +02:00
test_tokenization_funnel.py Reorganize repo (#8580) 2020-11-16 21:43:42 -05:00
test_tokenization_gpt2.py Enforce string-formatting with f-strings (#10980) 2021-03-31 10:00:27 -04:00
test_tokenization_herbert.py Reorganize repo (#8580) 2020-11-16 21:43:42 -05:00
test_tokenization_layoutlm.py make style (#11442) 2021-04-26 13:50:34 +02:00
test_tokenization_layoutlmv2.py Improve truncation_side (#14947) 2022-01-03 16:18:39 +01:00
test_tokenization_layoutxlm.py improve saving strategy of sentencepiece tokenizer (#15328) 2022-01-27 16:24:51 +01:00
test_tokenization_luke.py Feature/fix slow test in mluke (#14749) 2021-12-22 06:35:59 -05:00
test_tokenization_lxmert.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_tokenization_m2m_100.py [M2M100Tokenizer] fix _build_translation_inputs (#14382) 2021-11-13 20:57:12 +05:30
test_tokenization_marian.py Add regression tests for slow sentencepiece tokenizers. (#11737) 2021-06-01 09:24:39 -04:00
test_tokenization_mbart.py improve saving strategy of sentencepiece tokenizer (#15328) 2022-01-27 16:24:51 +01:00
test_tokenization_mbart50.py [Tests] Speed up tokenizer tests (#14964) 2021-12-28 17:02:50 +01:00
test_tokenization_mluke.py [Tests] Speed up tokenizer tests (#14964) 2021-12-28 17:02:50 +01:00
test_tokenization_mpnet.py [MPNet] Add slow to fast tokenizer converter (#9233) 2020-12-21 15:41:34 +01:00
test_tokenization_openai.py Make OpenAIGPTTokenizer work with SpaCy 2.x and 3.x (#15019) 2022-01-10 07:53:20 -05:00
test_tokenization_pegasus.py Add regression tests for slow sentencepiece tokenizers. (#11737) 2021-06-01 09:24:39 -04:00
test_tokenization_perceiver.py Fix tests (#14703) 2021-12-09 08:32:35 -05:00
test_tokenization_phobert.py Feature to use the PreTrainedTokenizerFast class as a stand-alone tokenizer (#11810) 2021-06-14 11:58:44 +02:00
test_tokenization_prophetnet.py Deprecate prepare_seq2seq_batch (#10287) 2021-02-22 12:36:16 -05:00
test_tokenization_rag.py Remove datasets requirement (#14795) 2021-12-16 14:34:14 -05:00
test_tokenization_realm.py Add FastTokenizer to REALM (#15211) 2022-01-19 15:19:36 +01:00
test_tokenization_reformer.py Add regression tests for slow sentencepiece tokenizers. (#11737) 2021-06-01 09:24:39 -04:00
test_tokenization_roberta.py update the arguments add_prefix_space and trim_offsets in backend_tokenizer.post_processor of RobertaTokenizerFast (#14752) 2021-12-22 10:51:55 +01:00
test_tokenization_roformer.py [RoFormer] Fix some issues (#12397) 2021-07-06 03:31:57 -04:00
test_tokenization_small_blenderbot.py Feature to use the PreTrainedTokenizerFast class as a stand-alone tokenizer (#11810) 2021-06-14 11:58:44 +02:00
test_tokenization_speech_to_text_2.py [Speech2Text2] Enable tokenizers (#14390) 2021-11-15 16:34:11 +01:00
test_tokenization_speech_to_text.py Add regression tests for slow sentencepiece tokenizers. (#11737) 2021-06-01 09:24:39 -04:00
test_tokenization_squeezebert.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_tokenization_t5.py Change how "additional_special_tokens" argument in the ".from_pretrained" method of the tokenizer is taken into account (#13056) 2021-08-23 14:35:18 +02:00
test_tokenization_tapas.py Improve truncation_side (#14947) 2022-01-03 16:18:39 +01:00
test_tokenization_transfo_xl.py Refactor prepare_seq2seq_batch (#9524) 2021-01-12 18:19:38 -05:00
test_tokenization_utils.py Patch recursive import (#11812) 2021-05-21 06:50:01 -04:00
test_tokenization_wav2vec2_phoneme.py Wav2Vec2 meets phonemes (#14353) 2021-12-17 19:56:44 +01:00
test_tokenization_wav2vec2.py **encode_plus() shouldn't run for W2V2CTC (#12655) 2021-07-13 06:31:56 -04:00
test_tokenization_xglm.py Add XGLM models (#14876) 2022-01-28 18:55:23 +01:00
test_tokenization_xlm_prophetnet.py Add regression tests for slow sentencepiece tokenizers. (#11737) 2021-06-01 09:24:39 -04:00
test_tokenization_xlm_roberta.py [Tests] Speed up tokenizer tests (#14964) 2021-12-28 17:02:50 +01:00
test_tokenization_xlm.py make style (#11442) 2021-04-26 13:50:34 +02:00
test_tokenization_xlnet.py Add regression tests for slow sentencepiece tokenizers. (#11737) 2021-06-01 09:24:39 -04:00
test_trainer_callback.py Proper import for unittest.mock.patch (#13085) 2021-08-12 11:23:00 +02:00
test_trainer_distributed.py Update namespaces inside torch.utils.data to the latest. (#13167) 2021-08-19 14:29:51 +02:00
test_trainer_seq2seq.py Remove datasets requirement (#14795) 2021-12-16 14:34:14 -05:00
test_trainer_tpu.py [TPU tests] Enable first TPU examples pytorch (#14121) 2021-10-28 01:22:28 +02:00
test_trainer_utils.py Allow dataset to be an optional argument for (Distributed)LengthGroupedSampler (#13820) 2021-10-05 09:04:39 -04:00
test_trainer.py Deprecates AdamW and adds --optim (#14744) 2022-01-13 08:14:51 -08:00
test_utils_check_copies.py Enable readme link synchronization (#13785) 2021-09-29 11:18:59 -04:00
test_versions_utils.py rm require_version_examples (#12088) 2021-06-09 11:02:52 -07:00