sgugger
5636cbb25d
Extra )
2020-09-14 09:37:55 -04:00
Sylvain Gugger
ccc8e30c8a
Clean up autoclass doc ( #7081 )
2020-09-14 09:26:41 -04:00
Stas Bekman
3ca1874ca4
[examples testing] restore code ( #7099 )
...
For some reason https://github.com/huggingface/transformers/pull/5512 re-added temp dir creation code that was removed by
https://github.com/huggingface/transformers/pull/6494 defeating the purpose of that PR for those tests.
2020-09-14 08:54:23 -04:00
Stas Bekman
4d39148419
fix deprecation warnings ( #7033 )
...
* fix deprecation warnings
* remove tests/test_tokenization_common.py's test_padding_to_max_length
* revert test_padding_to_max_length
2020-09-14 07:51:19 -04:00
Stas Bekman
576eec98e0
ignore FutureWarning in tests ( #7079 )
2020-09-14 07:50:51 -04:00
Bartosz Telenczuk
15d18e0307
fix link to paper ( #7116 )
2020-09-14 07:43:40 -04:00
Lysandre Debut
bb3106f741
Temporarily skip failing tests due to dependency change ( #7118 )
...
* Temporarily skip failing tests due to dependency change
* Remove trace
2020-09-14 07:42:13 -04:00
Sam Shleifer
0fab39695a
[s2s distill] allow pegasus-12-12 ( #7104 )
2020-09-14 00:03:59 -04:00
Sam Shleifer
de9e297964
[s2s] distributed eval cleanup ( #7110 )
2020-09-13 23:40:38 -04:00
Sam Shleifer
54395d87a6
Update xsum length penalty to better values ( #7107 )
2020-09-13 20:48:47 -04:00
Sam Shleifer
e7f8d2ab64
[s2s] two stage run_distributed_eval.py ( #7105 )
2020-09-13 17:28:18 -04:00
Sam Shleifer
0ec63afec2
fix bug in pegasus converter ( #7094 )
2020-09-13 15:11:47 -04:00
Sam Shleifer
b76cb1c3df
[s2s] run_eval supports --prefix clarg. ( #6953 )
2020-09-12 01:08:21 -04:00
李明浩
563ffb3dc3
Create README.md ( #7066 )
2020-09-11 15:21:05 -04:00
李明浩
1ad49cde3a
Create README.md ( #7067 )
2020-09-11 15:20:54 -04:00
Sagor Sarker
4753816e39
added bangla-bert-base model card and also modified other model cards ( #7071 )
...
* added bangla-bert-base
* Apply suggestions from code review
Co-authored-by: Julien Chaumond <chaumond@gmail.com>
2020-09-11 15:17:25 -04:00
Suraj Patil
0a8c17d53c
[T5Tokenizer] remove prefix_tokens ( #7078 )
2020-09-11 14:18:45 -04:00
Sylvain Gugger
4cbd50e611
Compute loss method ( #7074 )
2020-09-11 12:06:31 -04:00
Sylvain Gugger
ae736163d0
Add tests and fix various bugs in ModelOutput ( #7073 )
...
* Add tests and fix various bugs in ModelOutput
* Update tests/test_model_output.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2020-09-11 12:01:33 -04:00
Sylvain Gugger
e841b75dec
Automate the lists in auto-xxx docs ( #7061 )
...
* More readable dict
* More nlp -> datasets
* Revert "More nlp -> datasets"
This reverts commit 3cd1883d223cd1883d22
2020-09-11 10:42:09 -04:00
Sylvain Gugger
0054a48cdd
Add dep on datasets ( #7058 )
2020-09-11 04:43:19 -04:00
Patrick von Platen
221d4c63a3
clean naming ( #7068 )
2020-09-11 09:57:53 +02:00
Stas Bekman
8fcbe486e1
these tests require non-multigpu env ( #7059 )
...
* these tests require non-multigpu env
* cleanup
* clarify
2020-09-10 18:52:55 -04:00
Sam Shleifer
77950c485a
[wip/s2s] DistributedSortishSampler ( #7056 )
2020-09-10 15:23:44 -04:00
Sylvain Gugger
514486739c
Fix CI with change of name of nlp ( #7054 )
...
* nlp -> datasets
* More nlp -> datasets
* Woopsie
* More nlp -> datasets
* One last
2020-09-10 14:51:08 -04:00
Sam Shleifer
e9a2f772bc
[s2s] --eval_max_generate_length ( #7018 )
2020-09-10 14:11:34 -04:00
Stas Bekman
df4594a9da
[xlm tok] config dict: fix str into int to match definition ( #7034 )
2020-09-10 19:31:01 +02:00
Julien Chaumond
d6c08b07a0
[AutoTokenizer] Correct error message
2020-09-10 17:19:01 +02:00
Patrick von Platen
db38f7ce29
[BertGeneration, Docs] Fix another old name in docs ( #7050 )
...
* correct docs for bert generation
* upload
2020-09-10 17:12:33 +02:00
Patrick von Platen
3bd95b0faf
correct docs for bert generation ( #7048 )
2020-09-10 17:08:40 +02:00
Patrick von Platen
eb2feb5d90
Create README.md
2020-09-10 17:05:50 +02:00
Ashwin Geet Dsa
66a5a6fda8
fix to ensure that returned tensors after the tokenization is Long ( #7039 )
...
* fix to ensure that returned tensors after the tokenization is Long
* fix to ensure that returned tensors after the tokenization is Long
Co-authored-by: Ashwin Geet Dsa <adsa@grvingt-6.nancy.grid5000.fr>
2020-09-10 11:04:03 -04:00
Patrick von Platen
9ccdb1d517
Update README.md
2020-09-10 17:01:19 +02:00
Patrick von Platen
60698936fc
Create README.md
2020-09-10 17:00:10 +02:00
Patrick von Platen
e0c3bc8ee0
Create README.md
2020-09-10 16:51:15 +02:00
Patrick von Platen
c356b9878d
Create README.md
2020-09-10 16:45:44 +02:00
Patrick von Platen
5afd3f6196
Create README.md
2020-09-10 16:44:47 +02:00
Sylvain Gugger
15a189049e
Add TF Funnel Transformer ( #7029 )
...
* Add TF Funnel Transformer
* Proper dummy input
* Formatting
* Update src/transformers/modeling_tf_funnel.py
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
* Address review comments
* One review comment forgotten
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
2020-09-10 10:41:56 -04:00
Patrick von Platen
7fd1febf38
Add "Leveraging Pretrained Checkpoints for Generation" Seq2Seq models. ( #6594 )
...
* add conversion script
* improve conversion script
* make style
* add tryout files
* fix
* update
* add causal bert
* better names
* add tokenizer file as well
* finish causal_bert
* fix small bugs
* improve generate
* change naming
* renaming
* renaming
* renaming
* remove leftover files
* clean files
* add fix tokenizer
* finalize
* correct slow test
* update docs
* small fixes
* fix link
* adapt check repo
* apply sams and sylvains recommendations
* fix import
* implement Lysandres recommendations
* fix logger warn
2020-09-10 16:40:51 +02:00
Sylvain Gugger
d1691d90e5
Samell fixed in tf template ( #7044 )
2020-09-10 10:36:02 -04:00
Patrick von Platen
63e539459d
Update README.md
2020-09-10 16:34:28 +02:00
Patrick von Platen
054db06b1b
Create README.md
2020-09-10 16:30:46 +02:00
Lysandre Debut
b482ad474a
Fix template ( #7040 )
2020-09-10 08:45:52 -04:00
Yu Liu
762cba3bda
Albert pretrain datasets/ datacollator ( #6168 )
...
* add dataset for albert pretrain
* datacollator for albert pretrain
* naming, comprehension, file reading change
* data cleaning is no needed after this modification
* delete prints
* fix a bug
* file structure change
* add tests for albert datacollator
* remove random seed
* add back len and get item function
* sample file for testing and test code added
* format change for black
* more format change
* Style
* var assignment issue resolve
* add back wrongly deleted DataCollatorWithPadding in init file
* Style
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
Co-authored-by: Lysandre <lysandre.debut@reseau.eseo.fr>
2020-09-10 07:56:29 -04:00
Johann C. Rocholl
49e9be0639
Fix confusing warnings during TF2 import from PyTorch ( #6623 )
...
1. Swapped missing_keys and unexpected_keys.
2. Copy&paste error caused these warnings to say "from TF 2.0" when it's actually "from PyTorch".
2020-09-10 05:31:59 -04:00
Stas Bekman
4ee1053dcf
add -y to bypass prompt for transformers-cli upload ( #7035 )
2020-09-10 04:58:29 -04:00
Patrick von Platen
76818cc4c6
Create README.md
2020-09-09 16:26:35 +02:00
Lysandre Debut
15478c1287
Batch encore plus and overflowing tokens fails when non existing overflowing tokens for a sequence ( #6677 )
...
* Patch and test
* Fix tests
2020-09-09 06:55:17 -04:00
Henry Dashwood
9fd11bf1a8
replace torch.triu with onnx compatible code ( #6929 )
2020-09-09 04:56:40 -04:00
Julien Chaumond
ed71c21d6a
[from_pretrained] Allow tokenizer_type ≠ model_type ( #6995 )
2020-09-09 04:22:59 -04:00
