Patrick von Platen
31d06729f4
Update README.md
2021-07-20 14:19:37 +02:00
Patrick von Platen
2955d50e0c
[Longformer] Correct longformer docs ( #12809 )
...
* fix_torch_device_generate_test
* remove @
* correct longformer docs
Co-authored-by: Patrick von Platen <patrick@huggingface.co>
2021-07-20 14:17:21 +02:00
Patrick von Platen
13fefdf340
Update README.md
...
cc @patil-suraj
2021-07-20 13:51:15 +02:00
fgaim
66197adc98
Flax MLM: Allow validation split when loading dataset from local file ( #12689 )
...
* Allow validation split when loading dataset from local file
* Flax clm & t5, enable validation split for datasets loaded from local file
2021-07-20 13:38:25 +02:00
Will Rice
6f8e367ae9
Fix Padded Batch Error 12282 ( #12487 )
...
This fixes the padded batch [issue](https://github.com/huggingface/transformers/issues/12282 ). The error was generated due to the maximum sequence length of the attention mask not matching the padded sequence length of the hidden_states. `np.allclose` now passes with a 1e-2 absolute tolerance.
This change fixes
2021-07-20 13:36:47 +02:00
Stas Bekman
7fae535052
add troubleshooting docs ( #12791 )
2021-07-20 03:32:02 -04:00
Sylvain Gugger
0118ef89ee
Enforce eval and save strategies are compatible when --load_best_model_at_end ( #12786 )
...
* Enforce eval and save strategies are compatible when --load_best_model_at_end
* Update doc
* Fix typos
* Fix tests
2021-07-19 19:50:47 +02:00
Lysandre Debut
546dc24e08
Longer timeout for slow tests ( #12779 )
2021-07-19 04:55:40 -04:00
Antoni Baum
cab3b86892
[ray] Fix datasets_modules ImportError with Ray Tune ( #12749 )
...
* Fix dynamic_modules ImportError with Ray Tune
* Nit
2021-07-19 04:32:40 -04:00
Patrick von Platen
534f6eb9f1
Create README.md
2021-07-17 19:22:37 +02:00
Patrick von Platen
c6b9095cb2
Update README.md
2021-07-17 19:22:26 +02:00
Sylvain Gugger
da72ac6e26
Fix push_to_hub docstring and make it appear in doc ( #12770 )
2021-07-17 15:52:33 +02:00
Tomohiro Endo
08d609bfb8
Add tokenizers class mismatch detection between cls and checkpoint ( #12619 )
...
* Detect mismatch by analyzing config
* Fix comment
* Fix import
* Update src/transformers/tokenization_utils_base.py
Co-authored-by: SaulLu <55560583+SaulLu@users.noreply.github.com>
* Revise based on reviews
* remove kwargs
* Fix exception
* Fix handling exception again
* Disable mismatch test in PreTrainedTokenizerFast
Co-authored-by: SaulLu <55560583+SaulLu@users.noreply.github.com>
2021-07-17 15:52:21 +02:00
Patrick von Platen
b4b562d834
[Wav2Vec2] Padded vectors should not allowed to be sampled ( #12764 )
...
* fix_torch_device_generate_test
* remove @
* finish
* correct script
* correct script
2021-07-16 19:07:08 +02:00
SaulLu
6e87010060
Preserve list type of additional_special_tokens in special_token_map ( #12759 )
...
* preserve type of `additional_special_tokens` in `special_token_map`
* format
* Update src/transformers/tokenization_utils_base.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2021-07-16 18:26:54 +02:00
Funtowicz Morgan
fbf1397bf8
Turn on eval mode when exporting to ONNX ( #12758 )
...
* Set model in eval mode when exporting to ONNX.
* Disable t5 for now.
* Disable T5 with past too.
* Style.
2021-07-16 15:09:15 +02:00
Suraj Patil
8ef3f36561
fix typos ( #12757 )
2021-07-16 16:44:59 +05:30
Nathan Zhou
c07334c12e
add intel-tensorflow-avx512 to the candidates ( #12751 )
2021-07-16 05:54:49 -04:00
Stas Bekman
6989264963
[doc] testing: how to trigger a self-push workflow ( #12724 )
...
* [testing] details of how to start self-push workflow
* style
* fix
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2021-07-15 16:18:56 -07:00
Patrick von Platen
a76dd7ee82
Update README.md
2021-07-16 00:16:30 +01:00
Patrick von Platen
2e9fb13fb1
[Wav2Vec2] Correctly pad mask indices for PreTraining ( #12748 )
...
* fix_torch_device_generate_test
* remove @
* start adding tests
* correct wav2vec2 pretraining
* up
* up
Co-authored-by: Patrick von Platen <patrick@huggingface.co>
2021-07-15 21:40:25 +01:00
SaulLu
5f2791c7c1
Replace specific tokenizer in log message by AutoTokenizer ( #12745 )
2021-07-15 12:59:48 -04:00
Stas Bekman
31cfcbd3e2
[doc] performance: batch sizes ( #12725 )
2021-07-15 09:39:34 -07:00
Stas Bekman
68605e9db1
[doc] parallelism: Which Strategy To Use When ( #12712 )
2021-07-15 09:38:51 -07:00
Lysandre Debut
eb4d7ef97b
Remove framework mention ( #12731 )
2021-07-15 11:49:02 -04:00
Lysandre Debut
959d448b3f
Fix led torchscript ( #12735 )
...
* Don't test LED on torchscript
* Typo
2021-07-15 11:48:50 -04:00
Lysandre Debut
f03580fb02
Fix DETR integration test ( #12734 )
2021-07-15 11:48:37 -04:00
Lysandre Debut
f42d9dcc0e
Patch T5 device test ( #12742 )
2021-07-15 16:40:17 +01:00
Lysandre Debut
370be9cc38
Fix MBart failing test ( #12737 )
2021-07-15 16:39:35 +01:00
qqaatw
2349ac58c4
Translate README.md to Traditional Chinese ( #12701 )
...
* Add README_zh-tw.md
* Add links to each README.
* Fix a mismatched term.
* Minor improvements.
* Rename language code to be more inclusive.
* Polish terms to make them fluent.
* Remove redundant spaces.
* Fix typo.
2021-07-15 23:35:39 +08:00
Lysandre Debut
eb2e006b35
Skip test while the model is not available ( #12740 )
2021-07-15 09:14:12 -04:00
Lysandre Debut
8c7bd1b97b
Skip test while the model is not available ( #12739 )
2021-07-15 09:06:47 -04:00
Lysandre Debut
3290315a2a
Fix AutoModel tests ( #12733 )
2021-07-15 09:06:12 -04:00
Lysandre Debut
01cb2f25e3
LXMERT integration test typo ( #12736 )
2021-07-15 08:29:49 -04:00
Sylvain Gugger
199b4c5264
Init adds its own files as impacted ( #12709 )
2021-07-15 04:17:47 -04:00
Will Rice
6fb58d30b9
Fix typo in example ( #12716 )
2021-07-15 12:14:03 +05:30
Patrick von Platen
8244c5ad4f
[Flax] Correct shift labels for seq2seq models in Flax ( #12720 )
...
* fix_torch_device_generate_test
* remove @
* push
* fix marian
* fix
* up
2021-07-15 12:12:36 +05:30
Stas Bekman
1a3deae820
[trainer] release tmp memory in checkpoint load ( #12718 )
...
* [trainer] release tmp memory in checkpoint load
* Update src/transformers/trainer.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2021-07-14 15:18:02 -07:00
Stas Bekman
a18a17d2b6
[test] split test into 4 sub-tests to avoid timeout ( #12710 )
...
* split the test into 4 sub-tests to avoid timeout
* fix decorator order
2021-07-14 13:04:58 -07:00
Suraj Patil
44f5b260fe
flax model parallel training ( #12590 )
...
* update scripts
* add copyright
* add logging
* cleanup
* add z loss
* add readme
* shard description
* update readme
2021-07-14 22:55:44 +05:30
Matt
79c57e1a07
Deprecate TFTrainer ( #12706 )
...
* Deprecate TFTrainer
* Style pass
2021-07-14 15:59:14 +01:00
Sylvain Gugger
084873b025
Only test the files impacted by changes in the diff ( #12644 )
...
* Base test
* More test
* Fix mistake
* Add a docstring change
* Add doc ignore
* Add changes
* Add recursive dep search
* Add recursive dep search
* save
* Finalize test mapping
* Fix bug
* Print prettier
* Ignore comments and empty lines
* Make script runnable from anywhere
* Need dev install
* Like that
* Adapt
* Add as artifact
* Try on torch tests
* Fix yaml error
* Install GitPython
* Apply everywhere
* Be more defensive
* Revert to all tests if something is wrong
* Install GitPython
* Test if there are tests before launching.
* Fixes
* Fixes
* Fixes
* Fixes
* Bash syntax is horrible
* Be less stupid
* Try differently
* Typo
* Typo
* Typo
* Style
* Better name
* Escape quotes
* Ignore black unhelpful re-formatting
* Not a docstring
* Deal with inits in dependency map
* Run all tests once PR is merged.
* Add last job
* Apply suggestions from code review
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
* Stronger dependencies gather
* Ignore empty lines too!
* Clean up
* Fix quality
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
2021-07-14 10:56:55 -04:00
Funtowicz Morgan
11edecd753
Fix uninitialized variables when config.mask_feature_prob > 0 ( #12705 )
2021-07-14 15:30:19 +01:00
Matt
f9ac677eba
Update TF examples README ( #12703 )
...
* Update Transformers README, rename token_classification example to token-classification to be consistent with the others
* Update examples/tensorflow/README.md
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Add README for TF token classification
* Update examples/tensorflow/token-classification/README.md
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update examples/tensorflow/token-classification/README.md
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2021-07-14 15:15:25 +01:00
Patrick von Platen
f4399ec570
Update README.md
2021-07-14 12:54:31 +01:00
Funtowicz Morgan
d94773e685
Provide mask_time_indices to _mask_hidden_states to avoid double masking ( #12692 )
...
* We need to provide mask_time_indices to `_mask_hidden_states` to avoid applying the mask two times
* apply the same to wav2vec2
* Uniformize the style between hubert and wav2vec2
* fix tf as well
Co-authored-by: patrickvonplaten <patrick.v.platen@gmail.com>
2021-07-14 12:17:33 +01:00
Sylvain Gugger
144cea253f
Fix multiple choice doc examples ( #12679 )
2021-07-14 03:35:18 -04:00
Stas Bekman
5dd0c956a8
non-native optimizers are mostly ok with zero-offload ( #12690 )
2021-07-13 20:18:51 -07:00
yujun
4cdb7ee51d
fix #11724 ( #11897 )
2021-07-13 22:18:54 +01:00
Lysandre Debut
83f025125d
Add timeout to CI. ( #12684 )
...
* Global 60-300 seconds timeout
* Add verbose option
* [skip ci] typo
2021-07-13 15:13:18 -04:00
