Commit Graph

8821 Commits

Author SHA1 Message Date
Sylvain Gugger
95e2e14f9d
Revert to all tests whil we debug what's wrong (#13072) 2021-08-10 18:37:01 +02:00
Sylvain Gugger
477480ce2a Trigger GPU tests 2021-08-10 10:26:06 -04:00
Sylvain Gugger
0dad5d825d
Fix fallback of test_fetcher (#13071) 2021-08-10 16:17:06 +02:00
Sylvain Gugger
4dd857244c Merge branch 'master' of github.com:huggingface/transformers 2021-08-10 09:40:38 -04:00
Sylvain Gugger
bd5593b6c4 Try fecthing the last two commits 2021-08-10 09:40:16 -04:00
Sylvain Gugger
9e9b8f1d99
Roll out the test fetcher on push tests (#13055)
* Use test fetcher for push tests as well

* Force diff with last commit for circleCI on master

* Fix syntax error

* Style

* Schedule nightly tests
2021-08-10 14:54:52 +02:00
Sylvain Gugger
2e0d767ab2 Pin sacrebleu 2021-08-10 06:27:49 -04:00
Sylvain Gugger
0454e4bd8b
Fix ModelOutput instantiation form dictionaries (#13067)
* Fix ModelOutput instantiation form dictionaries

* Style
2021-08-10 12:20:04 +02:00
Aleksey Korshuk
3157fa3c53
docs: add HuggingArtists to community notebooks (#13050)
* Adding HuggingArtists to Community Notebooks

* Adding HuggingArtists to Community Notebooks

* Adding HuggingArtists to Community Notebooks

* docs: add HuggingArtists to community notebooks

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2021-08-10 09:36:44 +02:00
Kevin Canwen Xu
ab7551cd7f
Add try-except for torch_scatter (#13040)
* Add try-catch for torch_scatter

* Update modeling_tapas.py
2021-08-10 15:29:35 +08:00
SaulLu
76cadb7943
replace tgt_lang by tgt_text (#13061) 2021-08-09 22:47:05 +05:30
Lysandre
a8bf2fa76e Documentation for patch v4.9.2 2021-08-09 16:14:17 +02:00
Lysandre Debut
5008e08885
Add to ONNX docs (#13048)
* Add to ONNX docs

* Add MBART example

* Update docs/source/serialization.rst

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2021-08-09 09:51:49 -04:00
Lysandre Debut
6f5ab9daf1
Add MBART to models exportable with ONNX (#13049)
* Add MBART to models exportable with ONNX

* unittest mock

* Add tests

* Misc fixes
2021-08-09 08:56:04 -04:00
Patrick von Platen
13a9c9a354
[Flax] Refactor gpt2 & bert example docs (#13024)
* fix_torch_device_generate_test

* remove @

* improve docs for clm

* speed-ups

* correct t5 example as well

* push final touches

* Update examples/flax/language-modeling/README.md

* correct docs for mlm

* Update examples/flax/language-modeling/README.md

Co-authored-by: Patrick von Platen <patrick@huggingface.co>
2021-08-09 13:37:50 +02:00
abhishek thakur
3ff2cde5ca
tfhub.de -> tfhub.dev (#12565) 2021-08-09 08:11:17 +02:00
Patrick von Platen
24cbf6bc5a
Update README.md 2021-08-08 17:11:19 +02:00
lewtun
7390d9de63
Use min version for huggingface-hub dependency (#12961)
* Use min version for huggingface-hub dependency

* Update dependency version table
2021-08-08 09:06:05 -05:00
Sylvain Gugger
7fcee113c1
Tpu tie weights (#13030)
* Fix tied weights on TPU

* Manually tie weights in no trainer examples

* Fix for test

* One last missing

* Gettning owned by my scripts

* Address review comments

* Fix test

* Fix tests

* Fix reformer tests
2021-08-06 20:41:39 +02:00
Lysandre Debut
1bf38611a4
Put smaller ALBERT model (#13028) 2021-08-06 12:41:33 -04:00
Michael Benayoun
dc420b0eb1
T5 with past ONNX export (#13014)
T5 with past ONNX export, and more explicit past_key_values inputs and outputs names for ONNX model

Authored-by: Michael Benayoun <michael@huggingface.co>
2021-08-06 15:46:26 +02:00
Michael Benayoun
ee11224611
FX submodule naming fix (#13016)
Changed the way dynamically inserted submodules are named and the method used to insert them

Authored-by: Michael Benayoun <michael@huggingface.co>
2021-08-06 15:37:29 +02:00
Sylvain Gugger
9870093f7b
[WIP] Disentangle auto modules from other modeling files (#13023)
* Initial work

* All auto models

* All tf auto models

* All flax auto models

* Tokenizers

* Add feature extractors

* Fix typos

* Fix other typo

* Use the right config

* Remove old mapping names and update logic in AutoTokenizer

* Update check_table

* Fix copies and check_repo script

* Fix last test

* Add back name

* clean up

* Update template

* Update template

* Forgot a )

* Use alternative to fixup

* Fix TF model template

* Address review comments

* Address review comments

* Style
2021-08-06 13:12:30 +02:00
Patrick von Platen
2e4082364e
[Flax T5] Speed up t5 training (#13012)
* fix_torch_device_generate_test

* remove @

* update

* up

* fix

* remove f-stings

* correct readme

* up

Co-authored-by: Patrick von Platen <patrick@huggingface.co>
2021-08-06 11:21:37 +02:00
Patrick von Platen
60e448c87e
[Flax] Correct pt to flax conversion if from base to head (#13006)
* finish PR

* add tests

* correct tests

* finish

* correct other flax tests

* better naming

* correct naming

* finish

* apply sylvains suggestions
2021-08-05 18:38:50 +02:00
Nils Reimers
33929448a1
Replace // operator with / operator + long() (#13013) 2021-08-05 15:55:14 +02:00
Michael Benayoun
a6d62aaba0
GPT-Neo ONNX export (#12911)
GPT-Neo ONNX export and task / feature refactoring

Authored-by: Michael Benayoun <michael@huggingface.co>
2021-08-05 10:12:13 +02:00
Sasha Luccioni
8aa01d2a6d
Create perplexity.rst (#13004)
Updating the import for load_dataset
2021-08-05 02:56:13 -04:00
NielsRogge
83e5a10603
Add BEiT (#12994)
* First pass

* Make conversion script work

* Improve conversion script

* Fix bug, conversion script working

* Improve conversion script, implement BEiTFeatureExtractor

* Make conversion script work based on URL

* Improve conversion script

* Add tests, add documentation

* Fix bug in conversion script

* Fix another bug

* Add support for converting masked image modeling model

* Add support for converting masked image modeling

* Fix bug

* Add print statement for debugging

* Fix another bug

* Make conversion script finally work for masked image modeling models

* Move id2label for datasets to JSON files on the hub

* Make sure id's are read in as integers

* Add integration tests

* Make style & quality

* Fix test, add BEiT to README

* Apply suggestions from @sgugger's review

* Apply suggestions from code review

* Make quality

* Replace nielsr by microsoft in tests, add docs

* Rename BEiT to Beit

* Minor fix

* Fix docs of BeitForMaskedImageModeling

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2021-08-04 18:29:23 +02:00
Lysandre Debut
0dd1152c18
Skip ProphetNet test (#12462) 2021-08-04 18:24:54 +02:00
Arman Cohan
f82653874b
create tensors on device (#12846) 2021-08-04 17:58:30 +02:00
Patrick von Platen
fbf468b057
[Flax] Correct flax docs (#12782)
* fix_torch_device_generate_test

* remove @

* fix flax docs

* correct more docs in flax

* another correction

* fix flax docs

* Apply suggestions from code review
2021-08-04 16:31:23 +02:00
Patrick von Platen
a317e6c3be
[Flax] Correctly Add MT5 (#12988)
* finish PR

* finish mt5

* push

* up

* Update tests/test_modeling_flax_mt5.py

Co-authored-by: Suraj Patil <surajp815@gmail.com>

Co-authored-by: Suraj Patil <surajp815@gmail.com>
2021-08-04 16:03:13 +02:00
Patrick von Platen
da9754a3a0
[Flax] Align jax flax device name (#12987)
* [Flax] Align device name in docs

* make style

* fix import error
2021-08-04 16:00:09 +02:00
Aktsvigun
07df5578d9
pad_to_multiple_of added to DataCollatorForWholeWordMask (#12999)
* pad_to_multiple_of added to DataCollatorForWholeWordMask

* pad_to_multiple_of added to DataCollatorForWholeWordMask

Co-authored-by: Цвигун Аким Олегович <AOTsvigun@sberbank.ru>
2021-08-04 15:49:21 +02:00
Lysandre Debut
3f44a66cb6
Return raw outputs in TextClassificationPipeline (#8328)
* Return raw outputs in TextClassificationPipeline

* Style

* Support for problem type

* Update src/transformers/pipelines/text_classification.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Apply Nicolas' comments

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2021-08-04 08:42:47 -04:00
Sylvain Gugger
d4c834d2e0
Fix from_pretrained with corrupted state_dict (#12939)
* Fix from_pretrained with corrupted state_dict

* Adapt test

* Use better checkpoint

* Style

* Clean up
2021-08-04 11:48:39 +02:00
NielsRogge
a28da4c490
Replace nielsr by google namespace in tests (#12453) 2021-08-04 03:29:34 -04:00
Michal Szutenberg
f064e0a43d
Cast logits to fp32 at the end of TF_T5 (#12332)
This change enables tf.keras.mixed_precision with bf16
2021-08-03 20:02:59 +01:00
Philip May
b7439675b8
fix Trainer.train(resume_from_checkpoint=False) is causing an exception (#12981)
* fix #12970

* Update tests/test_trainer.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update tests/test_trainer.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update tests/test_trainer.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* remove unnecessary issue link

* fix test formatting

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2021-08-03 10:10:33 +02:00
Sylvain Gugger
790f1c9545
Fix template for inputs docstrings (#12976) 2021-08-03 08:28:25 +02:00
Chungman Lee
75b8990d90
fix typo in example/text-classification README (#12974)
* fix typo in example/text-classification README

* add space to align the table
2021-08-02 12:58:43 +02:00
Sylvain Gugger
c1a65385a1
Place BigBirdTokenizer in sentencepiece-only objects (#12975) 2021-08-02 08:26:38 +02:00
Tadej Svetina
b5995badc9
Fix typo in example of DPRReader (#12954) 2021-08-02 08:08:57 +02:00
Alex Hedges
a4340d3b85
Set tb_writer to None in TensorBoardCallback.on_train_end() (#12963) 2021-08-01 08:35:47 +02:00
Stefan Schweter
3d4b3bc3fd
examples: use correct way to get vocab size in flax lm readme (#12947) 2021-07-30 21:57:53 +05:30
Sylvain Gugger
23d6761f30
Fix division by zero in NotebookProgressPar (#12953) 2021-07-30 09:31:29 -04:00
Kevin Canwen Xu
8ff619d95e
Add multilingual documentation support (#12952)
* Add multilingual documentation support

* Add multilingual documentation support

* make style

* make style

* revert
2021-07-30 20:56:14 +08:00
wulu473
fe6ff4a920
Add substep callbacks (#12951)
Co-authored-by: Lukas Wutschitz <lukas.wutschitz@microsoft.com>
2021-07-30 08:20:38 -04:00
harshithapv
f84226b7a1
Log Azure ML metrics only for rank 0 (#12766)
* minor change to log azureml only for rank 0

* fix typo
2021-07-30 15:11:31 +08:00