Commit Graph

7528 Commits

Author SHA1 Message Date
Omar Sanseviero
8fe836af5a
Add Flax sprint project evaluation section (#12592) 2021-07-09 08:52:30 +02:00
Stas Bekman
ce111feed1
[doc] fix broken ref (#12597) 2021-07-08 14:11:01 -07:00
Stas Bekman
f0dde60127
[model.from_pretrained] raise exception early on failed load (#12574)
* [model.from_pretrained] raise exception early on failed load

Currently if `load` pretrained weights fails in `from_pretrained`, we first print a whole bunch of successful messages and then fail - this PR puts the exception first to avoid all the misleading messages.

* style

Co-authored-by: Suraj Patil <surajp815@gmail.com>
2021-07-08 08:17:51 -07:00
Sylvain Gugger
75e63dbf70
Fix MT5 init (#12591) 2021-07-08 11:12:18 -04:00
Nicolas Patry
4da568c152
Fixing the pipeline optimization by reindexing targets (V2) (#12330)
* Fixing the pipeline optimization by rescaling the logits first.

* Add test for target equivalence

Co-authored-by: Lysandre <lysandre.debut@reseau.eseo.fr>
2021-07-08 16:58:15 +02:00
Funtowicz Morgan
2aa3cd935d
[RFC] Laying down building stone for more flexible ONNX export capabilities (#11786)
* Laying down building stone for more flexible ONNX export capabilities

* Ability to provide a map of config key to override before exporting.

* Makes it possible to export BART with/without past keys.

* Supports simple mathematical syntax for OnnxVariable.repeated

* Effectively apply value override from onnx config for model

* Supports export with additional features such as with-past for seq2seq

* Store the output path directly in the args for uniform usage across.

* Make BART_ONNX_CONFIG_* constants and fix imports.

* Support BERT model.

* Use tokenizer for more flexibility in defining the inputs of a model.

* Add TODO as remainder to provide the batch/sequence_length as CLI args

* Enable optimizations to be done on the model.

* Enable GPT2 + past

* Improve model validation with outputs containing nested structures

* Enable Roberta

* Enable Albert

* Albert requires opset >= 12

* BERT-like models requires opset >= 12

* Remove double printing.

* Enable XLM-Roberta

* Enable DistilBERT

* Disable optimization by default

* Fix missing setattr when applying optimizer_features

* Add value field to OnnxVariable to define constant input (not from tokenizers)

* Add T5 support.

* Simplify model type retrieval

* Example exporting token_classification pipeline for DistilBERT.

* Refactoring to package `transformers.onnx`

* Solve circular dependency & __main__

* Remove unnecessary imports in `__init__`

* Licences

* Use @Narsil's suggestion to forward the model's configuration to the ONNXConfig to avoid interpolation.

* Onnx export v2 fixes (#12388)

* Tiny fixes
Remove `convert_pytorch` from onnxruntime-less runtimes
Correct reference to model

* Style

* Fix Copied from

* LongFormer ONNX config.

* Removed optimizations

* Remvoe bad merge relicas.

* Remove unused constants.

* Remove some deleted constants from imports.

* Fix unittest to remove usage of PyTorch model for onnx.utils.

* Fix distilbert export

* Enable ONNX export test for supported model.

* Style.

* Fix lint.

* Enable all supported default models.

* GPT2 only has one output

* Fix bad property name when overriding config.

* Added unittests and docstrings.

* Disable with_past tests for now.

* Enable outputs validation for default export.

* Remove graph opt lvls.

* Last commit with on-going past commented.

* Style.

* Disabled `with_past` for now

* Remove unused imports.

* Remove framework argument

* Remove TFPreTrainedModel reference

* Add documentation

* Add onnxruntime tests to CircleCI

* Add test

* Rename `convert_pytorch` to `export`

* Use OrderedDict for dummy inputs

* WIP Wav2Vec2

* Revert "WIP Wav2Vec2"

This reverts commit f665efb04c92525c3530e589029f0ae7afdf603e.

* Style

* Use OrderedDict for I/O

* Style.

* Specify OrderedDict documentation.

* Style :)

Co-authored-by: Lysandre <lysandre.debut@reseau.eseo.fr>
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
2021-07-08 10:54:42 -04:00
Sylvain Gugger
0085e712dd
Don't stop at num_epochs when using IterableDataset (#12561) 2021-07-08 07:24:46 -04:00
Sylvain Gugger
6f1adc4334
Fix group_lengths for short datasets (#12558) 2021-07-08 07:23:41 -04:00
Sylvain Gugger
0a6b9048d1
Init pickle (#12567)
* Try to pickle transformers

* Deal with special objs better

* Make picklable
2021-07-08 07:20:46 -04:00
Hwijeen Ahn
b29c394586
raise exception when arguments to pipeline are incomplete (#12548)
* raise exception when arguments are incomplete

* change exception to runtime error
2021-07-08 04:17:34 -04:00
Ibraheem Moosa
122d7dc34f
Remove logging of GPU count etc logging. (#12569)
Successfully logging this requires Pytorch. For the purposes of this script we are not using Pytorch.
2021-07-07 23:05:47 +01:00
Suraj Patil
d7e156bd1a
fix loading clip vision model (#12566) 2021-07-07 22:50:27 +05:30
Sylvain Gugger
b86826099b
Double check for attribute num_examples (#12562)
* Double check for attribute

* Use right name
2021-07-07 12:50:41 -04:00
Michal Szutenberg
0d2bffad31
Remove tf.roll wherever not needed (#12512)
It was used in shift_right.
After this change TF code is more similar to Pytorch implementations
Also, TF graphs are optimized (one node less)
2021-07-07 16:17:30 +01:00
Matt
95425d546d
Adding prepare_decoder_input_ids_from_labels methods to all ConditionalGeneration TF models (#12560) 2021-07-07 15:30:47 +01:00
Nicolas Patry
ebc69afc30
Adding support for pipeline("automatic-speech-recognition"). (#11525)
* Adding support for `pipeline("automatic-speech-recognition")`.

- Ugly `"config"` choice for AutoModel. It would be great to have the
possibility to have something like `AutoModelFor` that would implement
the same logic (Load the config, check Architectures and load the first
one)

* Remove `model_id` was not needed in the end.

* Rebased !

* Remove old code.

* Rename `nlp`.
2021-07-07 16:06:48 +02:00
Patrick von Platen
7d321b7689
[Flax] Allow retraining from save checkpoint (#12559)
* fix_torch_device_generate_test

* remove @

* finish
2021-07-07 19:13:43 +05:30
Souvic Chakraborty
1d6623c6a2
MLM training fails with no validation file(same as #12406 for pytorch now) (#12517)
* Validation split percentage to be used for custom data files also

Issue same as https://github.com/huggingface/transformers/issues/12406 fixed for pytorch branch run_mlm.py

* Validation split added in the right place

* Update run_clm.py

* validation split added for custom files

* Validation split added for custom files

* Update run_plm.py

* fixed validation split for custom files as input for pytorch examples in lm

* Update run_clm_no_trainer.py

* args modified
2021-07-07 09:05:44 -04:00
shabie
3488ef5a92
[trainer] add option to ignore keys for the train function too (#11719) (#12551) 2021-07-07 08:07:46 -04:00
Kevin Canwen Xu
45dcfdec52
Add a warning for broken ProphetNet fine-tuning (#12511) 2021-07-07 16:32:48 +08:00
Daniel Stancl
61400e1ec7
[Flax] Add FlaxMBart (#12236)
* Copy BART to MBart and rename some stuff

* Add copy statements pointing to FlaxBart

* Update/add some common files

* Update shift_tokens_rigth + fix imports

* Fix shift_tokens_right method according to MBart implementation

* Update shift_tokens_right in tests accordingly

* Fix the import issue and update docs file
* make style quality

* Do some minor changes according to patil-suraj suggestions

* Change the order of normalization layer and attention

* Add some copu statementes

* Update generate method and add integration test for mBart

* Make a few updates after a review

Besides, add `lang_code_to_id` to MBartTokenizeFast

* fix-copies; make style quality

* Apply suggestions from code review

* Apply suggestions from code review

* Apply suggestions from code review

* fix output type, style

* add copied from

* resolve conflicts

Co-authored-by: Suraj Patil <surajp815@gmail.com>
2021-07-07 12:20:38 +05:30
Suraj Patil
2d42915abe
[examples/flax] add adafactor optimizer (#12544)
* add adafactor

* Update examples/flax/language-modeling/run_mlm_flax.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2021-07-07 11:50:30 +05:30
Patrick von Platen
208df208bf
[Flax] Adapt examples to be able to use eval_steps and save_steps (#12543)
* fix_torch_device_generate_test

* remove @

* up

* up

* correct

* upload

Co-authored-by: Patrick von Platen <patrick@huggingface.co>
2021-07-06 19:41:51 +01:00
Lysandre
2870fd198f Bump CircleCI machine sizes 2021-07-06 17:46:39 +02:00
sadakmed
3fd85777ea
implementing tflxmertmodel integration test (#12497)
* implementing tflxmertmodel integration test

* move import

* revert and fix
2021-07-06 11:44:47 -04:00
SaulLu
09af5bdea3
Replace nn.Moudle by nn.Module (#12541) 2021-07-06 11:31:45 -04:00
Patrick von Platen
f42a0abf4b
Update README.md 2021-07-06 15:14:48 +01:00
Suzana Ilić
029b9d3f40
Update README (#12540) 2021-07-06 16:12:16 +02:00
Suraj Patil
7a259c190c
FlaxGPTNeo (#12493)
* flax gpt neo

* fix query scaling

* update generation test

* use flax model for test
2021-07-06 18:55:18 +05:30
yujun
626a0a0147
[RoFormer] Fix some issues (#12397)
* add RoFormerTokenizerFast into AutoTokenizer

* fix typo in roformer docs

* make onnx export happy

* update RoFormerConfig embedding_size

* use jieba not rjieba

* fix 12244 and make test_alignement passed

* update ARCHIVE_MAP

* make style & quality & fixup

* update

* make style & quality & fixup

* make style quality fixup

* update

* suggestion from LysandreJik

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* make style

* use rjieba

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
2021-07-06 03:31:57 -04:00
Suraj Patil
f5b0c1ecf0
[Flax] Fix hybrid clip (#12519)
* fix saving and loading

* update readme
2021-07-06 11:12:47 +05:30
Patrick von Platen
7d6285a921
[Wav2Vec2] Flax - Adapt wav2vec2 script (#12520)
* fix_torch_device_generate_test

* remove @

* adapt flax pretrain script
2021-07-05 23:49:47 +01:00
Patrick von Platen
4605b2b8ec
[Flax] Fix another bug in logging steps (#12516)
* fix_torch_device_generate_test

* remove @

* up
2021-07-05 18:35:22 +01:00
Patrick von Platen
d0f7508abe
[Flax] Correct logging steps flax (#12515)
* fix_torch_device_generate_test

* remove @

* push
2021-07-05 18:21:00 +01:00
Patrick von Platen
bb4ac2b5a8
[Flax] Correct flax training scripts (#12514)
* fix_torch_device_generate_test

* remove @

* add logging steps

* correct training scripts

* correct readme

* correct
2021-07-05 18:14:50 +01:00
Matt
ea55675024
NER example for Tensorflow (#12469)
* NER example for Tensorflow

* Style pass

* Style pass

* Added metric computation on the evaluation set

* Style pass

* Fixed label masking

* Style pass

* Style pass
2021-07-05 15:42:18 +01:00
Patrick von Platen
9b90810558
[Flax] Dataset streaming example (#12470)
* fix_torch_device_generate_test

* remove @

* upload

* finish dataset streaming

* adapt readme

* finish

* up

* up

* up

* up

* Apply suggestions from code review

* finish

* make style

* make style2

* finish

Co-authored-by: Patrick von Platen <patrick@huggingface.co>
2021-07-05 15:13:10 +01:00
Navjot
eceb1042c1
flax.linen.apply takes state as the first param, followed by the input (#12510) 2021-07-05 19:33:14 +05:30
Suraj Patil
f1c81d6b92
[Flax] ViT training example (#12300)
* begin script

* clean example, add readme

* update readme

* remove decay mask

* remove masking

* update readme & make flake happy
2021-07-05 18:23:03 +05:30
Akmal
e799e0f1ed
[Flax] Fix wav2vec2 pretrain arguments (#12498) 2021-07-05 13:35:20 +01:00
sadakmed
0e1718afb6
create LxmertModelIntegrationTest Pytorch (#9989)
* create LxmertModelIntegrationTest

* implementation using numpy seeding to fix inputs params.

* fix code quality

* isort check
2021-07-05 05:21:25 -04:00
Suraj Patil
23ab0b6980
[examples/flax] clip style image-text training example (#12491)
* clip style example

* fix post init

* add requirements

* update readme, few small fixes
2021-07-05 13:26:44 +05:30
Lysandre Debut
89a8739f0c
Add Repository import to the FLAX example script (#12501) 2021-07-05 03:51:11 -04:00
Patrick von Platen
2df63282e0
Update README.md 2021-07-04 13:16:29 +01:00
Omar Sanseviero
a76eebfc80
Add guide on how to build demos for the Flax sprint (#12468) 2021-07-02 20:35:17 +02:00
Patrick von Platen
b21905e03d
Update README.md 2021-07-02 14:12:47 +01:00
Patrick von Platen
d24a523130
Update README.md 2021-07-02 13:41:14 +01:00
Patrick von Platen
e3fce2f868
Update README.md
Thanks a lot @BirgerMoell
2021-07-02 12:12:54 +01:00
Lysandre Debut
b889d3f6c4
Fix TAPAS test uncovered by #12446 (#12480) 2021-07-02 04:35:10 -04:00
Matthew LeMay
b4ecc6bef2
fixed typo in flax-projects readme (#12466) 2021-07-02 12:27:39 +05:30