riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-31 02:02:21 +06:00
Code Issues Actions 33 Packages Projects Releases Wiki Activity
3a40cdf58d
transformers/tests
History
Thomas Wolf 3a40cdf58d
[tests|tokenizers] Refactoring pipelines test backbone - Small tokenizers improvements - General tests speedups (#7970) 
* WIP refactoring pipeline tests - switching to fast tokenizers

* fix dialog pipeline and fill-mask

* refactoring pipeline tests backbone

* make large tests slow

* fix tests (tf Bart inactive for now)

* fix doc...

* clean up for merge

* fixing tests - remove bart from summarization until there is TF

* fix quality and RAG

* Add new translation pipeline tests - fix JAX tests

* only slow for dialog

* Fixing the missing TF-BART imports in modeling_tf_auto

* spin out pipeline tests in separate CI job

* adding pipeline test to CI YAML

* add slow pipeline tests

* speed up tf and pt join test to avoid redoing all the standalone pt and tf tests

* Update src/transformers/tokenization_utils_base.py

Co-authored-by: Sam Shleifer <sshleifer@gmail.com>

* Update src/transformers/pipelines.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/pipelines.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Update src/transformers/testing_utils.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* add require_torch and require_tf in is_pt_tf_cross_test

Co-authored-by: Sam Shleifer <sshleifer@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
2020-10-23 15:58:19 +02:00
..
fixtures New run glue script (#7917) 2020-10-22 11:42:22 -04:00
__init__.py GPU text generation: mMoved the encoded_prompt to correct device 2020-01-06 15:11:12 +01:00
conftest.py [tests|tokenizers] Refactoring pipelines test backbone - Small tokenizers improvements - General tests speedups (#7970) 2020-10-23 15:58:19 +02:00
test_activations_tf.py Refactoring the TF activations functions (#7150) 2020-09-16 07:03:47 -04:00
test_activations.py Update repo to isort v5 (#6686) 2020-08-24 11:03:01 -04:00
test_benchmark_tf.py [Benchmarks] Change all args to from no_... to their positive form (#7075) 2020-09-23 13:25:24 -04:00
test_benchmark.py [Benchmarks] Change all args to from no_... to their positive form (#7075) 2020-09-23 13:25:24 -04:00
test_cli.py [transformers-cli] fix logger getter (#6777) 2020-08-27 20:01:17 -04:00
test_configuration_auto.py Move tests/utils.py -> transformers/testing_utils.py (#5350) 2020-07-01 10:31:17 -04:00
test_configuration_common.py [PretrainedConfig] Fix save pretrained config for edge case (#7943) 2020-10-22 15:39:01 +02:00
test_data_collator.py Mark big downloads slow (#7325) 2020-09-22 12:21:52 -04:00
test_doc_samples.py Move tests/utils.py -> transformers/testing_utils.py (#5350) 2020-07-01 10:31:17 -04:00
test_flax_auto.py Integrate Bert-like model on Flax runtime. (#3722) 2020-10-19 09:55:41 -04:00
test_hf_api.py Update repo to isort v5 (#6686) 2020-08-24 11:03:01 -04:00
test_hf_argparser.py parse arguments from dict (#4869) 2020-07-31 04:44:23 -04:00
test_logging.py adding TRANSFORMERS_VERBOSITY env var (#6961) 2020-09-09 04:08:01 -04:00
test_model_card.py GPU text generation: mMoved the encoded_prompt to correct device 2020-01-06 15:11:12 +01:00
test_model_output.py Add tests and fix various bugs in ModelOutput (#7073) 2020-09-11 12:01:33 -04:00
test_modeling_albert.py Black 20 release 2020-08-26 17:20:22 +02:00
test_modeling_auto.py Blenderbot (#7418) 2020-10-07 19:09:23 -04:00
test_modeling_bart.py Add TFBartForConditionalGeneration (#5411) 2020-10-21 13:10:16 +02:00
test_modeling_bert_generation.py Add "Leveraging Pretrained Checkpoints for Generation" Seq2Seq models. (#6594) 2020-09-10 16:40:51 +02:00
test_modeling_bert.py Black 20 release 2020-08-26 17:20:22 +02:00
test_modeling_blenderbot.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_modeling_camembert.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_modeling_common.py [multiple models] skip saving/loading deterministic state_dict keys (#7878) 2020-10-21 08:06:07 -04:00
test_modeling_ctrl.py Black 20 release 2020-08-26 17:20:22 +02:00
test_modeling_deberta.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_modeling_distilbert.py Black 20 release 2020-08-26 17:20:22 +02:00
test_modeling_dpr.py Black 20 release 2020-08-26 17:20:22 +02:00
test_modeling_electra.py Black 20 release 2020-08-26 17:20:22 +02:00
test_modeling_encoder_decoder.py ProphetNet (#7157) 2020-10-19 17:36:09 +02:00
test_modeling_flaubert.py Black 20 release 2020-08-26 17:20:22 +02:00
test_modeling_flax_bert.py Integrate Bert-like model on Flax runtime. (#3722) 2020-10-19 09:55:41 -04:00
test_modeling_flax_roberta.py Integrate Bert-like model on Flax runtime. (#3722) 2020-10-19 09:55:41 -04:00
test_modeling_fsmt.py [multiple models] skip saving/loading deterministic state_dict keys (#7878) 2020-10-21 08:06:07 -04:00
test_modeling_funnel.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_modeling_gpt2.py fix test (#7947) 2020-10-21 19:06:23 +02:00
test_modeling_layoutlm.py [testing] rename skip targets + docs (#7863) 2020-10-20 04:39:13 -04:00
test_modeling_longformer.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_modeling_lxmert.py fix (#6946) 2020-09-04 16:08:54 +02:00
test_modeling_marian.py [testing] slow tests should be marked as slow (#7895) 2020-10-22 06:34:05 -04:00
test_modeling_mbart.py [multiple models] skip saving/loading deterministic state_dict keys (#7878) 2020-10-21 08:06:07 -04:00
test_modeling_mobilebert.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_modeling_openai.py Gpt1 for sequence classification (#7683) 2020-10-13 05:06:15 -04:00
test_modeling_pegasus.py [multiple models] skip saving/loading deterministic state_dict keys (#7878) 2020-10-21 08:06:07 -04:00
test_modeling_prophetnet.py [PretrainedConfig] Fix save pretrained config for edge case (#7943) 2020-10-22 15:39:01 +02:00
test_modeling_rag.py [RAG] Propagating of n_docs as parameter to all RagModel's related functions (#7891) 2020-10-19 15:15:52 +02:00
test_modeling_reformer.py [testing] rename skip targets + docs (#7863) 2020-10-20 04:39:13 -04:00
test_modeling_roberta.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_modeling_squeezebert.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_modeling_t5.py ProphetNet (#7157) 2020-10-19 17:36:09 +02:00
test_modeling_tf_albert.py Update repo to isort v5 (#6686) 2020-08-24 11:03:01 -04:00
test_modeling_tf_auto.py Update repo to isort v5 (#6686) 2020-08-24 11:03:01 -04:00
test_modeling_tf_bart.py [tests|tokenizers] Refactoring pipelines test backbone - Small tokenizers improvements - General tests speedups (#7970) 2020-10-23 15:58:19 +02:00
test_modeling_tf_bert.py Custom TF weights loading (#7422) 2020-10-05 09:58:45 -04:00
test_modeling_tf_camembert.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_modeling_tf_common.py [tests|tokenizers] Refactoring pipelines test backbone - Small tokenizers improvements - General tests speedups (#7970) 2020-10-23 15:58:19 +02:00
test_modeling_tf_ctrl.py Black 20 release 2020-08-26 17:20:22 +02:00
test_modeling_tf_distilbert.py test_tf_common: remove un_used mixin class parameters (#6866) 2020-09-02 10:54:40 -04:00
test_modeling_tf_electra.py Black 20 release 2020-08-26 17:20:22 +02:00
test_modeling_tf_flaubert.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_modeling_tf_funnel.py Fix saving TF custom models (#7291) 2020-09-22 09:31:13 -04:00
test_modeling_tf_gpt2.py [Seq2Seq] Fix a couple of bugs and clean examples (#7474) 2020-10-01 17:38:50 +02:00
test_modeling_tf_longformer.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_modeling_tf_lxmert.py Adding the LXMERT pretraining model (MultiModal languageXvision) to HuggingFace's suite of models (#5793) 2020-09-03 04:02:25 -04:00
test_modeling_tf_mobilebert.py Update repo to isort v5 (#6686) 2020-08-24 11:03:01 -04:00
test_modeling_tf_openai.py [Generate] Facilitate PyTorch generate using ModelOutputs (#6735) 2020-09-01 12:38:25 +02:00
test_modeling_tf_pytorch.py [tests|tokenizers] Refactoring pipelines test backbone - Small tokenizers improvements - General tests speedups (#7970) 2020-10-23 15:58:19 +02:00
test_modeling_tf_roberta.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_modeling_tf_t5.py Add TFBartForConditionalGeneration (#5411) 2020-10-21 13:10:16 +02:00
test_modeling_tf_transfo_xl.py test_tf_common: remove un_used mixin class parameters (#6866) 2020-09-02 10:54:40 -04:00
test_modeling_tf_xlm_roberta.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_modeling_tf_xlm.py Black 20 release 2020-08-26 17:20:22 +02:00
test_modeling_tf_xlnet.py test_tf_common: remove un_used mixin class parameters (#6866) 2020-09-02 10:54:40 -04:00
test_modeling_transfo_xl.py [testing] rename skip targets + docs (#7863) 2020-10-20 04:39:13 -04:00
test_modeling_xlm_prophetnet.py ProphetNet (#7157) 2020-10-19 17:36:09 +02:00
test_modeling_xlm_roberta.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_modeling_xlm.py Black 20 release 2020-08-26 17:20:22 +02:00
test_modeling_xlnet.py [Seq2Seq] Fix a couple of bugs and clean examples (#7474) 2020-10-01 17:38:50 +02:00
test_onnx.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_optimization_tf.py Update repo to isort v5 (#6686) 2020-08-24 11:03:01 -04:00
test_optimization.py Format 2020-08-27 18:31:51 +02:00
test_pipelines_common.py [tests|tokenizers] Refactoring pipelines test backbone - Small tokenizers improvements - General tests speedups (#7970) 2020-10-23 15:58:19 +02:00
test_pipelines_conversational.py [tests|tokenizers] Refactoring pipelines test backbone - Small tokenizers improvements - General tests speedups (#7970) 2020-10-23 15:58:19 +02:00
test_pipelines_dialog.py [tests|tokenizers] Refactoring pipelines test backbone - Small tokenizers improvements - General tests speedups (#7970) 2020-10-23 15:58:19 +02:00
test_pipelines_feature_extraction.py [tests|tokenizers] Refactoring pipelines test backbone - Small tokenizers improvements - General tests speedups (#7970) 2020-10-23 15:58:19 +02:00
test_pipelines_fill_mask.py [tests|tokenizers] Refactoring pipelines test backbone - Small tokenizers improvements - General tests speedups (#7970) 2020-10-23 15:58:19 +02:00
test_pipelines_ner.py [tests|tokenizers] Refactoring pipelines test backbone - Small tokenizers improvements - General tests speedups (#7970) 2020-10-23 15:58:19 +02:00
test_pipelines_question_answering.py [tests|tokenizers] Refactoring pipelines test backbone - Small tokenizers improvements - General tests speedups (#7970) 2020-10-23 15:58:19 +02:00
test_pipelines_sentiment_analysis.py [tests|tokenizers] Refactoring pipelines test backbone - Small tokenizers improvements - General tests speedups (#7970) 2020-10-23 15:58:19 +02:00
test_pipelines_summarization.py [tests|tokenizers] Refactoring pipelines test backbone - Small tokenizers improvements - General tests speedups (#7970) 2020-10-23 15:58:19 +02:00
test_pipelines_text_generation.py [tests|tokenizers] Refactoring pipelines test backbone - Small tokenizers improvements - General tests speedups (#7970) 2020-10-23 15:58:19 +02:00
test_pipelines_text2text_generation.py [tests|tokenizers] Refactoring pipelines test backbone - Small tokenizers improvements - General tests speedups (#7970) 2020-10-23 15:58:19 +02:00
test_pipelines_translation.py [tests|tokenizers] Refactoring pipelines test backbone - Small tokenizers improvements - General tests speedups (#7970) 2020-10-23 15:58:19 +02:00
test_pipelines_zero_shot.py [tests|tokenizers] Refactoring pipelines test backbone - Small tokenizers improvements - General tests speedups (#7970) 2020-10-23 15:58:19 +02:00
test_retrieval_rag.py Allow Custom Dataset in RAG Retriever (#7763) 2020-10-19 19:42:45 +02:00
test_skip_decorators.py [testing] rename skip targets + docs (#7863) 2020-10-20 04:39:13 -04:00
test_tokenization_albert.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_tokenization_auto.py [testing] slow tests should be marked as slow (#7895) 2020-10-22 06:34:05 -04:00
test_tokenization_bart.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_tokenization_bert_generation.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_tokenization_bert_japanese.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_tokenization_bert.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_tokenization_bertweet.py Add new pre-trained models BERTweet and PhoBERT (#6129) 2020-09-18 13:16:43 -04:00
test_tokenization_blenderbot.py Blenderbot (#7418) 2020-10-07 19:09:23 -04:00
test_tokenization_camembert.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_tokenization_common.py [tests|tokenizers] Refactoring pipelines test backbone - Small tokenizers improvements - General tests speedups (#7970) 2020-10-23 15:58:19 +02:00
test_tokenization_ctrl.py Adding Fast tokenizers for SentencePiece based tokenizers - Breaking: remove Transfo-XL fast tokenizer (#7141) 2020-10-08 11:32:16 +02:00
test_tokenization_deberta.py Add DeBERTa model (#5929) 2020-09-30 07:07:30 -04:00
test_tokenization_distilbert.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_tokenization_dpr.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_tokenization_fsmt.py [fsmt test] basic config test with online model + super tiny model (#7860) 2020-10-22 09:14:54 -04:00
test_tokenization_funnel.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_tokenization_gpt2.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_tokenization_herbert.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_tokenization_layoutlm.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_tokenization_lxmert.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_tokenization_marian.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_tokenization_mbart.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_tokenization_openai.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_tokenization_pegasus.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_tokenization_phobert.py Add new pre-trained models BERTweet and PhoBERT (#6129) 2020-09-18 13:16:43 -04:00
test_tokenization_prophetnet.py ProphetNet (#7157) 2020-10-19 17:36:09 +02:00
test_tokenization_rag.py [Rag] Fix loading of pretrained Rag Tokenizer (#7756) 2020-10-13 14:34:22 +02:00
test_tokenization_reformer.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_tokenization_roberta.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_tokenization_squeezebert.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_tokenization_t5.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_tokenization_transfo_xl.py Adding Fast tokenizers for SentencePiece based tokenizers - Breaking: remove Transfo-XL fast tokenizer (#7141) 2020-10-08 11:32:16 +02:00
test_tokenization_utils.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_tokenization_xlm_prophetnet.py ProphetNet (#7157) 2020-10-19 17:36:09 +02:00
test_tokenization_xlm_roberta.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_tokenization_xlm.py Adding Fast tokenizers for SentencePiece based tokenizers - Breaking: remove Transfo-XL fast tokenizer (#7141) 2020-10-08 11:32:16 +02:00
test_tokenization_xlnet.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_trainer_callback.py Only log total_flos at the end of training (#7981) 2020-10-22 14:26:55 -04:00
test_trainer_distributed.py Add predict step accumulation (#7767) 2020-10-14 11:41:45 -04:00
test_trainer_tpu.py Add predict step accumulation (#7767) 2020-10-14 11:41:45 -04:00
test_trainer_utils.py Add predict step accumulation (#7767) 2020-10-14 11:41:45 -04:00
test_trainer.py TensorBoard/Wandb/optuna/raytune integration improvements. (#7935) 2020-10-21 17:18:52 +02:00
test_utils_check_copies.py Get a better error when check_copies fails (#7457) 2020-09-30 10:05:14 +02:00