riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-31 02:02:21 +06:00
Code Issues Actions 33 Packages Projects Releases Wiki Activity
0282e24eef
transformers/tests
History
Mansi Mane 0282e24eef
Smmp batch not divisible by microbatches fix (#10778) 
* Added debug prints

* Added config

* Added prints

* Added prints

* Added extra samples to SequentialDistributedSampler

* Added extra samples to SequentialDistributedSampler

Updated SequentialDistributedSampler call

* Added deubg prints

* Removed extra prints

* Making predicitons and labels multiple of batchsize

* updated number of microbatches

* Removed extra prints

* Made start_remainder similar to DistributedSamplerWithLoop

* Minor spacing update

* Added debug prints

Added config

Added prints

Added prints

* Added extra samples to SequentialDistributedSampler

Updated SequentialDistributedSampler call

Added extra samples to SequentialDistributedSampler

Added deubg prints

Removed extra prints

Making predicitons and labels multiple of batchsize

updated number of microbatches

Removed extra prints

Squashing redundant commits

* Made start_remainder similar to DistributedSamplerWithLoop

Minor spacing update

Made start_remainder similar to DistributedSamplerWithLoop

* Test and styling

* Rename test

Co-authored-by: Sylvain Gugger <sylvain.gugger@gmail.com>
2021-03-17 19:18:11 -04:00
..
fixtures Conversion from slow to fast for BPE spm vocabs contained an error. (#10120) 2021-02-13 08:24:53 -05:00
__init__.py GPU text generation: mMoved the encoded_prompt to correct device 2020-01-06 15:11:12 +01:00
conftest.py Flax testing should not run the full torch test suite (#10725) 2021-03-16 08:05:37 +03:00
test_activations_tf.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_activations.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_benchmark_tf.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_benchmark.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_cli.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_configuration_auto.py Reorganize repo (#8580) 2020-11-16 21:43:42 -05:00
test_configuration_common.py [PretrainedConfig] Fix save pretrained config for edge case (#7943) 2020-10-22 15:39:01 +02:00
test_data_collator.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_doc_samples.py Fix ignore list behavior in doctests (#8213) 2020-11-02 08:47:37 -05:00
test_feature_extraction_common.py [FeatureExtractorSavingUtils] Refactor PretrainedFeatureExtractor (#10594) 2021-03-09 12:16:59 +03:00
test_feature_extraction_speech_to_text.py Speech2TextTransformer (#10175) 2021-03-10 21:42:04 +05:30
test_feature_extraction_wav2vec2.py W2v2 test require torch (#10665) 2021-03-11 12:56:12 -05:00
test_file_utils.py Patches the full import failure and adds a test (#10750) 2021-03-16 15:37:52 -04:00
test_flax_auto.py [ci, flax] non-existing models are unlikely to pass tests (#10409) 2021-02-26 12:35:36 +03:00
test_generation_beam_search.py Add flags to return scores, hidden states and / or attention weights in GenerationMixin (#9150) 2021-01-06 17:11:42 +01:00
test_generation_logits_process.py remove adjust_logits_during_generation method (#10087) 2021-02-10 22:39:09 +05:30
test_generation_stopping_criteria.py Adding new parameter to generate: max_time. (#9846) 2021-03-12 10:11:50 +01:00
test_generation_utils.py Adding new parameter to generate: max_time. (#9846) 2021-03-12 10:11:50 +01:00
test_hf_api.py [hf_api] delete deprecated methods and tests (2) 2021-02-12 21:46:17 +01:00
test_hf_argparser.py Adding required flags to non-default arguments in hf_argparser (#10688) 2021-03-15 09:27:55 -04:00
test_logging.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_model_card.py GPU text generation: mMoved the encoded_prompt to correct device 2020-01-06 15:11:12 +01:00
test_model_output.py Add tests and fix various bugs in ModelOutput (#7073) 2020-09-11 12:01:33 -04:00
test_modeling_albert.py Fix integration slow tests (#10670) 2021-03-11 13:43:53 -05:00
test_modeling_auto.py LayoutLM Config (#9539) 2021-01-12 10:03:50 -05:00
test_modeling_bart.py fix bart tests (#10060) 2021-02-08 13:25:09 +03:00
test_modeling_bert_generation.py Implementing the test integration of BertGeneration (#9990) 2021-02-08 08:22:19 -05:00
test_modeling_bert.py Fix integration slow tests (#10670) 2021-03-11 13:43:53 -05:00
test_modeling_blenderbot_small.py BartForCausalLM analogs to ProphetNetForCausalLM (#9128) 2021-02-04 11:56:12 +03:00
test_modeling_blenderbot.py Hotfixing tests (blenderbot decoderonly tests, also need to remove (#10003) 2021-02-04 11:41:34 -05:00
test_modeling_bort.py ADD BORT (#9813) 2021-01-27 21:25:11 +03:00
test_modeling_camembert.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_modeling_common.py Add Fine-Tuning for Wav2Vec2 (#10145) 2021-03-01 12:13:17 +03:00
test_modeling_convbert.py Fix integration slow tests (#10670) 2021-03-11 13:43:53 -05:00
test_modeling_ctrl.py Ctrl for sequence classification (#8812) 2020-12-01 09:49:27 +01:00
test_modeling_deberta_v2.py Fix integration slow tests (#10670) 2021-03-11 13:43:53 -05:00
test_modeling_deberta.py Fix integration slow tests (#10670) 2021-03-11 13:43:53 -05:00
test_modeling_distilbert.py Fix integration slow tests (#10670) 2021-03-11 13:43:53 -05:00
test_modeling_dpr.py Fix slow dpr test (#10059) 2021-02-08 04:43:25 -05:00
test_modeling_electra.py Fix integration slow tests (#10670) 2021-03-11 13:43:53 -05:00
test_modeling_encoder_decoder.py BartForCausalLM analogs to ProphetNetForCausalLM (#9128) 2021-02-04 11:56:12 +03:00
test_modeling_flaubert.py Integration test for FlauBert (#10022) 2021-02-08 04:36:50 -05:00
test_modeling_flax_bert.py [Flax] Align FlaxBertForMaskedLM with BertForMaskedLM, implement from_pretrained, init (#9054) 2020-12-16 13:03:32 +01:00
test_modeling_flax_common.py Flax testing should not run the full torch test suite (#10725) 2021-03-16 08:05:37 +03:00
test_modeling_flax_roberta.py [Flax] Align FlaxBertForMaskedLM with BertForMaskedLM, implement from_pretrained, init (#9054) 2020-12-16 13:03:32 +01:00
test_modeling_fsmt.py Fix torch 1.8.0 segmentation fault (#10546) 2021-03-05 12:10:19 -05:00
test_modeling_funnel.py Switch return_dict to True by default. (#8530) 2020-11-16 11:43:00 -05:00
test_modeling_gpt2.py GPT2DoubleHeadsModel made parallelizable (#10658) 2021-03-15 09:10:44 -04:00
test_modeling_ibert.py I-BERT model support (#10153) 2021-02-25 10:06:42 -05:00
test_modeling_layoutlm.py Fix slow tests v4.2.0 (#9561) 2021-01-13 09:55:48 -05:00
test_modeling_led.py Add head_mask and decoder_head_mask to PyTorch LED (#9856) 2021-02-02 11:06:52 -08:00
test_modeling_longformer.py Add head_mask and decoder_head_mask to PyTorch LED (#9856) 2021-02-02 11:06:52 -08:00
test_modeling_lxmert.py Remove redundant test_head_masking = True flags in test files (#9858) 2021-01-28 10:09:13 -05:00
test_modeling_m2m_100.py layerdrop 0 (#10604) 2021-03-09 17:35:07 +03:00
test_modeling_marian.py Conversion to tensors requires padding (#10661) 2021-03-11 12:58:15 -05:00
test_modeling_mbart.py Fix integration slow tests (#10670) 2021-03-11 13:43:53 -05:00
test_modeling_mobilebert.py Removed unused encoder_hidden_states and encoder_attention_mask (#8972) 2020-12-08 12:04:34 -05:00
test_modeling_mpnet.py Add MP Net 2 (#9004) 2020-12-09 10:32:43 -05:00
test_modeling_mt5.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_modeling_openai.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_modeling_pegasus.py BartForCausalLM analogs to ProphetNetForCausalLM (#9128) 2021-02-04 11:56:12 +03:00
test_modeling_prophetnet.py up (#10771) 2021-03-17 16:15:14 +03:00
test_modeling_rag.py make rag tests smaller (#10679) 2021-03-15 10:07:12 +03:00
test_modeling_reformer.py Add flags to return scores, hidden states and / or attention weights in GenerationMixin (#9150) 2021-01-06 17:11:42 +01:00
test_modeling_roberta.py Add caching mechanism to BERT, RoBERTa (#9183) 2020-12-23 23:01:32 +05:30
test_modeling_speech_to_text.py Speech2TextTransformer (#10175) 2021-03-10 21:42:04 +05:30
test_modeling_squeezebert.py Fix integration slow tests (#10670) 2021-03-11 13:43:53 -05:00
test_modeling_t5.py Fix torch 1.8.0 segmentation fault (#10546) 2021-03-05 12:10:19 -05:00
test_modeling_tapas.py Remove tolerance + drop_rows_to_fit by default (#9507) 2021-01-11 08:02:41 -05:00
test_modeling_tf_albert.py Add AMP for Albert (#10141) 2021-02-15 17:18:33 +01:00
test_modeling_tf_auto.py Optional layers (#8961) 2020-12-08 09:14:09 -05:00
test_modeling_tf_bart.py Making TF BART-like models XLA and AMP compliant (#10191) 2021-02-17 17:48:56 +01:00
test_modeling_tf_bert.py Check TF ops for ONNX compliance (#10025) 2021-02-15 07:55:10 -05:00
test_modeling_tf_blenderbot_small.py Making TF BART-like models XLA and AMP compliant (#10191) 2021-02-17 17:48:56 +01:00
test_modeling_tf_blenderbot.py TensorFlow tests: having from_pt set to True requires torch to be installed. (#10664) 2021-03-12 14:16:40 +03:00
test_modeling_tf_bort.py ADD BORT (#9813) 2021-01-27 21:25:11 +03:00
test_modeling_tf_camembert.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_modeling_tf_common.py Tests run on Docker (#10681) 2021-03-15 17:28:01 -04:00
test_modeling_tf_convbert.py ConvBERT fix torch <> tf weights conversion (#10314) 2021-02-24 14:55:34 +03:00
test_modeling_tf_ctrl.py Make TF CTRL compliant with XLA and AMP (#10209) 2021-02-17 18:54:15 +01:00
test_modeling_tf_distilbert.py Check TF ops for ONNX compliance (#10025) 2021-02-15 07:55:10 -05:00
test_modeling_tf_dpr.py Check TF ops for ONNX compliance (#10025) 2021-02-15 07:55:10 -05:00
test_modeling_tf_electra.py Check TF ops for ONNX compliance (#10025) 2021-02-15 07:55:10 -05:00
test_modeling_tf_flaubert.py Making TF XLM-like models XLA and AMP compliant (#10211) 2021-02-17 18:02:48 +01:00
test_modeling_tf_funnel.py Fix AMP (#10216) 2021-02-18 06:29:43 -05:00
test_modeling_tf_gpt2.py Making TF GPT2 compliant with XLA and AMP (#10230) 2021-02-18 09:36:01 +01:00
test_modeling_tf_led.py Making TF Longformer-like models compliant with AMP (#10233) 2021-02-22 15:41:56 +01:00
test_modeling_tf_longformer.py Making TF Longformer-like models compliant with AMP (#10233) 2021-02-22 15:41:56 +01:00
test_modeling_tf_lxmert.py Making TF Lxmert model compliant with AMP (#10257) 2021-02-19 06:54:14 -05:00
test_modeling_tf_marian.py TensorFlow tests: having from_pt set to True requires torch to be installed. (#10664) 2021-03-12 14:16:40 +03:00
test_modeling_tf_mbart.py Deprecate prepare_seq2seq_batch (#10287) 2021-02-22 12:36:16 -05:00
test_modeling_tf_mobilebert.py Making TF MobileBert model compliant with AMP (#10259) 2021-02-19 06:55:25 -05:00
test_modeling_tf_mpnet.py Making TF MPNet model compliant with XLA (#10260) 2021-02-19 06:56:41 -05:00
test_modeling_tf_mt5.py Adjust loss difference (#10669) 2021-03-12 09:09:46 +03:00
test_modeling_tf_openai.py Making TF OpenAI GPT model compliant with AMP and XLA (#10261) 2021-02-19 09:33:25 -05:00
test_modeling_tf_pegasus.py Fixes Pegasus tokenization tests (#10671) 2021-03-11 13:35:50 -05:00
test_modeling_tf_pytorch.py Optional layers (#8961) 2020-12-08 09:14:09 -05:00
test_modeling_tf_rag.py make rag tests smaller (#10679) 2021-03-15 10:07:12 +03:00
test_modeling_tf_roberta.py Check TF ops for ONNX compliance (#10025) 2021-02-15 07:55:10 -05:00
test_modeling_tf_t5.py Fix XLA and AMP (#10262) 2021-02-19 06:57:16 -05:00
test_modeling_tf_transfo_xl.py Making TF TransfoXL model compliant with AMP (#10264) 2021-02-19 06:58:07 -05:00
test_modeling_tf_xlm_roberta.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_modeling_tf_xlm.py Making TF XLM-like models XLA and AMP compliant (#10211) 2021-02-17 18:02:48 +01:00
test_modeling_tf_xlnet.py Check TF ops for ONNX compliance (#10025) 2021-02-15 07:55:10 -05:00
test_modeling_transfo_xl.py Add flags to return scores, hidden states and / or attention weights in GenerationMixin (#9150) 2021-01-06 17:11:42 +01:00
test_modeling_wav2vec2.py small improvements (#10773) 2021-03-17 18:10:17 +03:00
test_modeling_xlm_prophetnet.py Ci test tf super slow (#8007) 2020-10-30 10:25:48 -04:00
test_modeling_xlm_roberta.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_modeling_xlm.py Add flags to return scores, hidden states and / or attention weights in GenerationMixin (#9150) 2021-01-06 17:11:42 +01:00
test_modeling_xlnet.py Add flags to return scores, hidden states and / or attention weights in GenerationMixin (#9150) 2021-01-06 17:11:42 +01:00
test_offline.py offline mode for firewalled envs (part 2) (#10569) 2021-03-08 08:52:20 -08:00
test_onnx.py Onnx fix test (#10663) 2021-03-11 13:38:29 -05:00
test_optimization_tf.py Use stable functions (#9369) 2021-01-05 03:58:26 -05:00
test_optimization.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_pipelines_common.py [PretrainedFeatureExtractor] + Wav2Vec2FeatureExtractor, Wav2Vec2Processor, Wav2Vec2Tokenizer (#10324) 2021-02-25 17:42:46 +03:00
test_pipelines_conversational.py Fixing conversation test for torch 1.8 (#10545) 2021-03-05 09:24:14 -05:00
test_pipelines_feature_extraction.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_pipelines_fill_mask.py Adding skip_special_tokens=True to FillMaskPipeline (#9783) 2021-01-26 10:06:28 +01:00
test_pipelines_ner.py Fixing NER pipeline for list inputs. (#10184) 2021-02-15 06:22:45 -05:00
test_pipelines_question_answering.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_pipelines_sentiment_analysis.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_pipelines_summarization.py Fix torch 1.8.0 segmentation fault (#10546) 2021-03-05 12:10:19 -05:00
test_pipelines_table_question_answering.py Adding a test to prevent late failure in the Table question answering (#9808) 2021-01-27 04:10:53 -05:00
test_pipelines_text_generation.py Adding a new return_full_text parameter to TextGenerationPipeline. (#9852) 2021-01-29 10:27:32 +01:00
test_pipelines_text2text_generation.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_pipelines_translation.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_pipelines_zero_shot.py zero-shot pipeline multi_class -> multi_label (#10727) 2021-03-15 16:02:46 -06:00
test_processor_speech_to_text.py Speech2TextTransformer (#10175) 2021-03-10 21:42:04 +05:30
test_processor_wav2vec2.py [PretrainedFeatureExtractor] + Wav2Vec2FeatureExtractor, Wav2Vec2Processor, Wav2Vec2Tokenizer (#10324) 2021-02-25 17:42:46 +03:00
test_retrieval_rag.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_sequence_feature_extraction_common.py Speech2TextTransformer (#10175) 2021-03-10 21:42:04 +05:30
test_skip_decorators.py [testing] rename skip targets + docs (#7863) 2020-10-20 04:39:13 -04:00
test_tokenization_albert.py ALBERT Tokenizer integration test (#9943) 2021-02-02 04:39:33 -05:00
test_tokenization_auto.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_tokenization_bart.py Deprecate prepare_seq2seq_batch (#10287) 2021-02-22 12:36:16 -05:00
test_tokenization_barthez.py Speedup tf tests (#10601) 2021-03-08 21:44:07 -05:00
test_tokenization_bert_generation.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_tokenization_bert_japanese.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_tokenization_bert.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_tokenization_bertweet.py Reorganize repo (#8580) 2020-11-16 21:43:42 -05:00
test_tokenization_blenderbot.py [PyTorch Bart] Split Bart into different models (#9343) 2021-01-05 22:00:05 +01:00
test_tokenization_camembert.py Conversion from slow to fast for BPE spm vocabs contained an error. (#10120) 2021-02-13 08:24:53 -05:00
test_tokenization_common.py Flax testing should not run the full torch test suite (#10725) 2021-03-16 08:05:37 +03:00
test_tokenization_ctrl.py Refactor prepare_seq2seq_batch (#9524) 2021-01-12 18:19:38 -05:00
test_tokenization_deberta_v2.py Integrate DeBERTa v2(the 1.5B model surpassed human performance on Su… (#10018) 2021-02-19 18:34:44 -05:00
test_tokenization_deberta.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_tokenization_distilbert.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_tokenization_dpr.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_tokenization_fsmt.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_tokenization_funnel.py Reorganize repo (#8580) 2020-11-16 21:43:42 -05:00
test_tokenization_gpt2.py [Tokenizer Utils Base] Make pad function more flexible (#9928) 2021-02-02 10:35:27 +03:00
test_tokenization_herbert.py Reorganize repo (#8580) 2020-11-16 21:43:42 -05:00
test_tokenization_layoutlm.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_tokenization_lxmert.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_tokenization_m2m_100.py Add m2m100 (#10236) 2021-03-06 22:14:16 +05:30
test_tokenization_marian.py Flax testing should not run the full torch test suite (#10725) 2021-03-16 08:05:37 +03:00
test_tokenization_mbart.py Make Barthez tokenizer tests a bit faster (#10399) 2021-02-25 11:42:25 -05:00
test_tokenization_mbart50.py Make Barthez tokenizer tests a bit faster (#10399) 2021-02-25 11:42:25 -05:00
test_tokenization_mpnet.py [MPNet] Add slow to fast tokenizer converter (#9233) 2020-12-21 15:41:34 +01:00
test_tokenization_openai.py [Tokenizer Utils Base] Make pad function more flexible (#9928) 2021-02-02 10:35:27 +03:00
test_tokenization_pegasus.py Deprecate prepare_seq2seq_batch (#10287) 2021-02-22 12:36:16 -05:00
test_tokenization_phobert.py Reorganize repo (#8580) 2020-11-16 21:43:42 -05:00
test_tokenization_prophetnet.py Deprecate prepare_seq2seq_batch (#10287) 2021-02-22 12:36:16 -05:00
test_tokenization_rag.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_tokenization_reformer.py [Tokenizer Utils Base] Make pad function more flexible (#9928) 2021-02-02 10:35:27 +03:00
test_tokenization_roberta.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_tokenization_small_blenderbot.py [PyTorch Bart] Split Bart into different models (#9343) 2021-01-05 22:00:05 +01:00
test_tokenization_speech_to_text.py Speech2TextTransformer (#10175) 2021-03-10 21:42:04 +05:30
test_tokenization_squeezebert.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_tokenization_t5.py Flax testing should not run the full torch test suite (#10725) 2021-03-16 08:05:37 +03:00
test_tokenization_tapas.py Fix slow tests v4.2.0 (#9561) 2021-01-13 09:55:48 -05:00
test_tokenization_transfo_xl.py Refactor prepare_seq2seq_batch (#9524) 2021-01-12 18:19:38 -05:00
test_tokenization_utils.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_tokenization_wav2vec2.py W2v2 test require torch (#10665) 2021-03-11 12:56:12 -05:00
test_tokenization_xlm_prophetnet.py Reorganize repo (#8580) 2020-11-16 21:43:42 -05:00
test_tokenization_xlm_roberta.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_tokenization_xlm.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_tokenization_xlnet.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_trainer_callback.py Fix tests of TrainerCallback (#10615) 2021-03-09 16:25:32 -05:00
test_trainer_distributed.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_trainer_seq2seq.py fix bert2bert test (#10063) 2021-02-08 16:04:28 +03:00
test_trainer_tpu.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_trainer_utils.py Smmp batch not divisible by microbatches fix (#10778) 2021-03-17 19:18:11 -04:00
test_trainer.py make failure to find a resume checkpoint fatal + tests (#10777) 2021-03-17 11:16:37 -07:00
test_utils_check_copies.py Check copies blackify (#10775) 2021-03-17 18:11:20 -04:00
test_versions_utils.py Copyright (#8970) 2020-12-07 18:36:34 -05:00