transformers/tests
Julien Plu f9414f7553
Tensorflow improvements (#4530)
* Better None gradients handling

* Apply Style

* Apply Style

* Create a loss class per task to compute its respective loss

* Add loss classes to the ALBERT TF models

* Add loss classes to the BERT TF models

* Add question answering and multiple choice to TF Camembert

* Remove prints

* Add multiple choice model to TF DistilBERT + loss computation

* Add question answering model to TF Electra + loss computation

* Add token classification, question answering and multiple choice models to TF Flaubert

* Add multiple choice model to TF Roberta + loss computation

* Add multiple choice model to TF XLM + loss computation

* Add multiple choice and question answering models to TF XLM-Roberta

* Add multiple choice model to TF XLNet + loss computation

* Remove unused parameters

* Add task loss classes

* Reorder TF imports + add new model classes

* Add new model classes

* Bugfix in TF T5 model

* Bugfix for TF T5 tests

* Bugfix in TF T5 model

* Fix TF T5 model tests

* Fix T5 tests + some renaming

* Fix inheritance issue in the AutoX tests

* Add tests for TF Flaubert and TF XLM Roberta

* Add tests for TF Flaubert and TF XLM Roberta

* Remove unused piece of code in the TF trainer

* bugfix and remove unused code

* Bugfix for TF 2.2

* Apply Style

* Divide TFSequenceClassificationAndMultipleChoiceLoss into their two respective name

* Apply style

* Mirror the PT Trainer in the TF one: fp16, optimizers and tb_writer as class parameter and better dataset handling

* Fix TF optimizations tests and apply style

* Remove useless parameter

* Bugfix and apply style

* Fix TF Trainer prediction

* Now the TF models return the loss such as their PyTorch couterparts

* Apply Style

* Ignore some tests output

* Take into account the SQuAD cls_index, p_mask and is_impossible parameters for the QuestionAnswering task models.

* Fix names for SQuAD data

* Apply Style

* Fix conflicts with 2.11 release

* Fix conflicts with 2.11

* Fix wrongname

* Add better documentation on the new create_optimizer function

* Fix isort

* logging_dir: use same default as PyTorch

Co-authored-by: Julien Chaumond <chaumond@gmail.com>
2020-06-04 19:45:53 -04:00
..
fixtures BIG Reorganize examples (#4213) 2020-05-07 13:48:44 -04:00
__init__.py GPU text generation: mMoved the encoded_prompt to correct device 2020-01-06 15:11:12 +01:00
test_activations.py Reformer (#3351) 2020-05-07 10:17:01 +02:00
test_benchmark.py [Benchmark] Memory benchmark utils (#4198) 2020-05-27 23:22:16 +02:00
test_configuration_auto.py Map configs to models and tokenizers 2020-01-13 23:11:44 +00:00
test_configuration_common.py Pass kwargs to configuration (#3147) 2020-03-05 17:16:57 -05:00
test_doc_samples.py make style 2020-03-04 20:18:07 -05:00
test_hf_api.py [cli] {login, upload, s3} display more helpful error messages 2020-04-30 12:51:06 -04:00
test_hf_argparser.py [Benchmark] Memory benchmark utils (#4198) 2020-05-27 23:22:16 +02:00
test_model_card.py GPU text generation: mMoved the encoded_prompt to correct device 2020-01-06 15:11:12 +01:00
test_modeling_albert.py Kill model archive maps (#4636) 2020-06-02 09:39:33 -04:00
test_modeling_auto.py Kill model archive maps (#4636) 2020-06-02 09:39:33 -04:00
test_modeling_bart.py Fix CI after killing archive maps (#4724) 2020-06-02 10:21:09 -04:00
test_modeling_bert.py Kill model archive maps (#4636) 2020-06-02 09:39:33 -04:00
test_modeling_camembert.py Fix slow gpu tests lysandre (#4487) 2020-05-20 11:59:45 -04:00
test_modeling_common.py Kill model archive maps (#4636) 2020-06-02 09:39:33 -04:00
test_modeling_ctrl.py Kill model archive maps (#4636) 2020-06-02 09:39:33 -04:00
test_modeling_distilbert.py Kill model archive maps (#4636) 2020-06-02 09:39:33 -04:00
test_modeling_electra.py Kill model archive maps (#4636) 2020-06-02 09:39:33 -04:00
test_modeling_encoder_decoder.py Unify label args (#4722) 2020-06-03 09:36:26 -04:00
test_modeling_flaubert.py Kill model archive maps (#4636) 2020-06-02 09:39:33 -04:00
test_modeling_gpt2.py Kill model archive maps (#4636) 2020-06-02 09:39:33 -04:00
test_modeling_longformer.py [Longformer] Better handling of global attention mask vs local attention mask (#4672) 2020-05-29 17:58:42 +02:00
test_modeling_marian.py [MarianTokenizer] implement save_vocabulary and other common methods (#4389) 2020-05-19 19:45:49 -04:00
test_modeling_openai.py Kill model archive maps (#4636) 2020-06-02 09:39:33 -04:00
test_modeling_reformer.py [Reformer] Improved memory if input is shorter than chunk length (#4720) 2020-06-02 23:08:39 +02:00
test_modeling_roberta.py Kill model archive maps (#4636) 2020-06-02 09:39:33 -04:00
test_modeling_t5.py Kill model archive maps (#4636) 2020-06-02 09:39:33 -04:00
test_modeling_tf_albert.py Kill model archive maps (#4636) 2020-06-02 09:39:33 -04:00
test_modeling_tf_auto.py Kill model archive maps (#4636) 2020-06-02 09:39:33 -04:00
test_modeling_tf_bert.py Kill model archive maps (#4636) 2020-06-02 09:39:33 -04:00
test_modeling_tf_camembert.py Add camembert integration tests (#3375) 2020-03-24 10:18:37 +01:00
test_modeling_tf_common.py Tensorflow improvements (#4530) 2020-06-04 19:45:53 -04:00
test_modeling_tf_ctrl.py Kill model archive maps (#4636) 2020-06-02 09:39:33 -04:00
test_modeling_tf_distilbert.py Kill model archive maps (#4636) 2020-06-02 09:39:33 -04:00
test_modeling_tf_electra.py Kill model archive maps (#4636) 2020-06-02 09:39:33 -04:00
test_modeling_tf_flaubert.py Tensorflow improvements (#4530) 2020-06-04 19:45:53 -04:00
test_modeling_tf_gpt2.py Kill model archive maps (#4636) 2020-06-02 09:39:33 -04:00
test_modeling_tf_openai_gpt.py Kill model archive maps (#4636) 2020-06-02 09:39:33 -04:00
test_modeling_tf_roberta.py TFRobertaModelIntegrationTest requires tf (#4726) 2020-06-02 12:59:00 -04:00
test_modeling_tf_t5.py [ci] Load pretrained models into the default (long-lived) cache 2020-04-30 22:30:15 -04:00
test_modeling_tf_transfo_xl.py Kill model archive maps (#4636) 2020-06-02 09:39:33 -04:00
test_modeling_tf_xlm_roberta.py Tensorflow improvements (#4530) 2020-06-04 19:45:53 -04:00
test_modeling_tf_xlm.py Kill model archive maps (#4636) 2020-06-02 09:39:33 -04:00
test_modeling_tf_xlnet.py Kill model archive maps (#4636) 2020-06-02 09:39:33 -04:00
test_modeling_transfo_xl.py Kill model archive maps (#4636) 2020-06-02 09:39:33 -04:00
test_modeling_xlm_roberta.py [ci] Re-run integration ground truth from fairseq 2020-03-03 15:31:40 -05:00
test_modeling_xlm.py Kill model archive maps (#4636) 2020-06-02 09:39:33 -04:00
test_modeling_xlnet.py Kill model archive maps (#4636) 2020-06-02 09:39:33 -04:00
test_onnx.py Fix onnx export input names order (#4641) 2020-06-01 16:12:48 +02:00
test_optimization_tf.py Tensorflow improvements (#4530) 2020-06-04 19:45:53 -04:00
test_optimization.py GPU text generation: mMoved the encoded_prompt to correct device 2020-01-06 15:11:12 +01:00
test_pipelines.py [test_pipelines] Mark tests > 10s @slow, small speedups (#4421) 2020-05-18 12:23:21 -04:00
test_tokenization_albert.py [cleanup] test_tokenization_common.py (#4390) 2020-05-19 10:46:55 -04:00
test_tokenization_auto.py Test correct tokenizers after default switch (#3003) 2020-02-24 18:45:53 -05:00
test_tokenization_bert_japanese.py Kill model archive maps (#4636) 2020-06-02 09:39:33 -04:00
test_tokenization_bert.py [cleanup] test_tokenization_common.py (#4390) 2020-05-19 10:46:55 -04:00
test_tokenization_common.py Introduce a new tensor type for return_tensors on tokenizer for NumPy (#4585) 2020-06-04 06:57:01 +02:00
test_tokenization_ctrl.py 💄 super 2020-01-15 18:33:50 -05:00
test_tokenization_distilbert.py [cleanup] test_tokenization_common.py (#4390) 2020-05-19 10:46:55 -04:00
test_tokenization_fast.py Fix add_special_tokens on fast tokenizers (#4531) 2020-05-28 10:54:45 -04:00
test_tokenization_gpt2.py Tokenizers v3.0.0 (#3185) 2020-04-07 00:29:15 +02:00
test_tokenization_marian.py [MarianTokenizer] implement save_vocabulary and other common methods (#4389) 2020-05-19 19:45:49 -04:00
test_tokenization_openai.py [cleanup] test_tokenization_common.py (#4390) 2020-05-19 10:46:55 -04:00
test_tokenization_roberta.py Fix slow gpu tests lysandre (#4487) 2020-05-20 11:59:45 -04:00
test_tokenization_t5.py [cleanup] test_tokenization_common.py (#4390) 2020-05-19 10:46:55 -04:00
test_tokenization_transfo_xl.py 💄 super 2020-01-15 18:33:50 -05:00
test_tokenization_utils.py Introduce a new tensor type for return_tensors on tokenizer for NumPy (#4585) 2020-06-04 06:57:01 +02:00
test_tokenization_xlm_roberta.py [cleanup] test_tokenization_common.py (#4390) 2020-05-19 10:46:55 -04:00
test_tokenization_xlm.py [cleanup] test_tokenization_common.py (#4390) 2020-05-19 10:46:55 -04:00
test_tokenization_xlnet.py [cleanup] test_tokenization_common.py (#4390) 2020-05-19 10:46:55 -04:00
test_trainer_distributed.py Update test_trainer_distributed.py 2020-05-20 18:26:51 -04:00
test_trainer.py Adds predict stage for glue tasks, and generate result files which can be submitted to gluebenchmark.com (#4463) 2020-05-21 09:17:44 -04:00
utils.py Fix nn.DataParallel compatibility in PyTorch 1.5 (#4300) 2020-05-18 20:34:50 -04:00