riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-31 02:02:21 +06:00
Code Issues Actions 23 Packages Projects Releases Wiki Activity
5,593 Commits 887 Branches 232 Tags 2.1 GiB
64b4d25cf3
Commit Graph

5593 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Stas Bekman
64b4d25cf3
[fsmt test] basic config test with online model + super tiny model (#7860) 
* basic config test with online model

* typo

* style

* better test
 2020-10-22 09:14:54 -04:00
Julien Chaumond
3479787edc
Disable inference API for t5-11b (#7978) 2020-10-22 09:08:37 -04:00
Julien Chaumond
a7db81c33f [model_card] t5-11b move disclaimer to top of page 
cc @Narsil @patrickvonplaten
 2020-10-22 14:35:31 +02:00
Haebin Shin
f774b2e8c4
support relative path for best_model_checkpoint (#7973) 2020-10-22 07:55:31 -04:00
Stas Bekman
8348105692
[testing] slow tests should be marked as slow (#7895) 
* slow tests should be slow

* exception note

* style

* integrate LysandreJik's notes with some expansions

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* another slow test

* fix link, and prose

* clarify.

* note from Sam

* typo

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2020-10-22 06:34:05 -04:00
rmroczkowski
95792a948e
Herbert tokenizer auto load (#7968) 2020-10-22 05:48:29 -04:00
zolekode
4abb7ffc18
added qg evaluation notebook (#7958) 
* added qg evaluation notebook

* Update notebooks/README.md

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2020-10-22 11:02:12 +02:00
Stas Bekman
8b38173398
[seq2seq testing] multigpu test run via subprocess (#7281) 
Co-authored-by: Sam Shleifer <sshleifer@gmail.com>
 2020-10-21 17:20:53 -04:00
Julien Chaumond
f8d3695e8c
[model_cards] camembert: dataset = oscar 
Hat/tip @pjox
 2020-10-21 14:17:56 -04:00
Evan Pete Walsh
16da877139
fix 'encode_plus' docstring for 'special_tokens_mask' (0s and 1s were reversed) (#7949) 
* fix docstring for 'special_tokens_mask'

* revert auto formatter changes

* revert another auto format

* revert another auto format
 2020-10-21 13:57:44 -04:00
Patrick von Platen
52decab371
fix test (#7947) 2020-10-21 19:06:23 +02:00
Patrick von Platen
9b6610f7f6
[ProphetNet] Correct Doc string example (#7944) 
* correct xlm prophetnet auto model and examples

* fix line-break docs
 2020-10-21 17:27:20 +02:00
François Lagunas
e174bfeb34
TensorBoard/Wandb/optuna/raytune integration improvements. (#7935) 
Improved TensorBoard and Wandb integration, as well as optuna and ray/tune support, with minor modifications to trainer core code.
 2020-10-21 17:18:52 +02:00
Ali Hamdi Ali Fadel
bf162ce8ca
Add AI-SOCO models (#7867) 2020-10-21 09:24:43 -04:00
Fangyu Liu
58fb25f25b
Create README.md (#7857) 
* Create README.md

model card for cambridgeltl/BioRedditBERT-uncased.

* Update model_cards/cambridgeltl/BioRedditBERT-uncased/README.md

Co-authored-by: Julien Chaumond <chaumond@gmail.com>
 2020-10-21 08:41:41 -04:00
Manuel Romero
2b07ec7823
Model card for German BERT fine-tuned for LER/NER (#7855) 2020-10-21 08:31:41 -04:00
MichalPleban
35d2ad5b83
Create README.md (#7819) 2020-10-21 08:30:01 -04:00
Wuwei Lan
bdda4f2249
Create README.md (#7625) 
* Create README.md

* Update model_cards/lanwuwei/GigaBERT-v3-Arabic-and-English/README.md

* Update model_cards/lanwuwei/GigaBERT-v3-Arabic-and-English/README.md

Co-authored-by: Julien Chaumond <chaumond@gmail.com>
 2020-10-21 08:29:39 -04:00
Manuel Romero
8e23749649
Add missing comma (#7870) 2020-10-21 08:24:12 -04:00
Manuel Romero
3eaa007d78
Create README.md (#7899) 2020-10-21 08:23:55 -04:00
Julien Chaumond
758572cad8 [model_cards] move hatmimoha/arabic-ner to correct location 
see 16d3cc187d and https://github.com/huggingface/transformers/pull/7836
 2020-10-21 14:13:17 +02:00
Stas Bekman
57516c0cc8
[multiple models] skip saving/loading deterministic state_dict keys (#7878) 
* make the save_load special key tests common

* handle mbart

* cleaner solution

* fix

* move test_save_load_missing_keys back into fstm for now

* restore

* style

* add marian

* add pegasus

* blenderbot

* revert - no static embed
 2020-10-21 08:06:07 -04:00
quentinheinrich
006a16483f
update model cards of Illuin models (#7930) 2020-10-21 08:05:53 -04:00
hatmimoha
16d3cc187d
model card for arabic-ner model (#7836) 
* Create README.md

README file for the Arabic NER model

* Update README.md

* Update README.md

* Update hatmimoha/arabic-ner/README.md

Co-authored-by: Julien Chaumond <chaumond@gmail.com>
 2020-10-21 08:02:40 -04:00
Sam Shleifer
829842159e
Add TFBartForConditionalGeneration (#5411) 
* half done

* doc improvement

* Cp test file

* brokedn

* broken test

* undo some mess

* ckpt

* borked

* Halfway

* 6 passing

* boom boom

* Much progress but still 6

* boom boom

* merged master

* 10 passing

* boom boom

* Style

* no t5 changes

* 13 passing

* Integration test failing, but not gibberish

* Frustrated

* Merged master

* 4 fail

* 4 fail

* fix return_dict

* boom boom

* Still only 4

* prepare method

* prepare method

* before delete classif

* Skip tests to avoid adding boilerplate

* boom boom

* fast tests passing

* style

* boom boom

* Switch to supporting many input types

* remove FIXMENORM

* working

* Fixed past_key_values/decoder_cached_states confusion

* new broken test

* Fix attention mask kwarg name

* undo accidental

* Style and reviewers

* style

* Docs and common tests

* Cleaner assert messages

* copy docs

* style issues

* Sphinx fix

* Simplify caching logic

* test does not require torch

* copy _NoLayerEmbedTokens

* Update src/transformers/modeling_tf_bart.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Update tests/test_modeling_tf_bart.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Update src/transformers/modeling_tf_bart.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Update src/transformers/modeling_tf_bart.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Update src/transformers/modeling_tf_bart.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Line length and dont document None

* Add pipeline test coverage

* assert msg

* At parity

* Assert messages

* mark slow

* Update compile test

* back in init

* Merge master

* Fix tests

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
 2020-10-21 13:10:16 +02:00
Patrick von Platen
5cd9e2cba1
Update README.md 2020-10-21 12:43:42 +02:00
Patrick von Platen
220b5f97ca
Create README.md 2020-10-21 12:34:46 +02:00
Patrick von Platen
8ffd7fb12d
Update README.md 2020-10-21 12:27:09 +02:00
Patrick von Platen
613ab364eb
Update README.md 2020-10-21 12:23:17 +02:00
Patrick von Platen
f7eb17dc47
Update README.md 2020-10-21 12:19:44 +02:00
Patrick von Platen
29792864cb
[ProphetNet] Add Question Generation Model + Test (#7942) 
* new prophetnet model

* correct name

* make style
 2020-10-21 11:49:58 +02:00
Joe Davison
13842e413c
PPL guide minor code snippet fix (#7938) 2020-10-20 16:17:39 -06:00
Stas Bekman
0e24e4c136
[s2s] create doc for pegasus/fsmt replication (#7934) 2020-10-20 15:07:52 -04:00
Lysandre Debut
96f4828ace
Respect the 119 line chars (#7928) 2020-10-20 11:02:47 -04:00
Lysandre
ef0ac063c9 Docs for v3.4.0 2020-10-20 16:29:00 +02:00
Lysandre
eb0e0ce2ad Release: v3.4.0 2020-10-20 16:22:26 +02:00
Patrick von Platen
0264048660
Update README.md 2020-10-20 16:13:49 +02:00
Patrick von Platen
ffd675b42c
add summary (#7927) 2020-10-20 10:11:02 -04:00
Lysandre Debut
5547b40b13
labels and decoder_input_ids to Glossary (#7906) 
* labels and decoder_input_ids to Glossary

* Formatting fixes

* Update docs/source/glossary.rst

Co-authored-by: Sam Shleifer <sshleifer@gmail.com>

* sam's comments

Co-authored-by: Sam Shleifer <sshleifer@gmail.com>
 2020-10-20 09:50:47 -04:00
Patrick von Platen
f3312515b7
Add note for WikiSplit 2020-10-20 15:42:29 +02:00
Patrick von Platen
0724c0f3a2
Fix EncoderDecoder WikiSplit Example 2020-10-20 15:13:22 +02:00
Stas Bekman
ca37db0559
[flax] fix repo_check (#7914) 
* [flax] fix repo_check

Unless, this is actually a problem, this adds `modeling_flax_utils` to ignore list. otherwise currently it expects to have a 'tests/test_modeling_flax_utils.py' for it.
for context please see: https://github.com/huggingface/transformers/pull/3722#issuecomment-712360415

* fix 2 more issues

* merge https://github.com/huggingface/transformers/pull/7919/
 2020-10-20 07:55:40 -04:00
Shai Erera
048dd6cf10
Fix bug in _sorted_checkpoints (#7880) 
I'm using transformers 3.3.1 and run a training script with `--save_total_limit 3`. I hit the exception below, and after debugging the code found that it wrongly tries to index into the `best_model_checkpoint`'s *str* rather than the `sorted_checkpoints` array. When running without the fix I got this exception:

```
Traceback (most recent call last):
  File "/<HOME>/.conda/envs/transformers/lib/python3.7/site-packages/transformers/trainer.py", line 921, in _save_training
    self._rotate_checkpoints(use_mtime=True)
  File "/<HOME>/.conda/envs/transformers/lib/python3.7/site-packages/transformers/trainer.py", line 1283, in _rotate_checkpoints
    checkpoints_sorted = self._sorted_checkpoints(use_mtime=use_mtime)
  File "/<HOME>/.conda/envs/transformers/lib/python3.7/site-packages/transformers/trainer.py", line 1274, in _sorted_checkpoints
    checkpoints_sorted[best_model_index],
TypeError: 'str' object does not support item assignment
```
 2020-10-20 07:50:47 -04:00
Sylvain Gugger
6d4f8bd02a
Add Flax dummy objects (#7918) 2020-10-20 07:45:48 -04:00
Stas Bekman
3e31e7f956
[testing] rename skip targets + docs (#7863) 
* rename skip targets + docs

* fix quotes

* style

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* small improvements

* fix

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2020-10-20 04:39:13 -04:00
Patrick von Platen
c912ba5f69
[EncoderDecoder] Fix Typo (#7915) 
* fix encoder decoder models

* add .gitignore
 2020-10-19 22:02:42 +02:00
Bram Vanroy
55bcd0cb59
Raise error when using AMP on non-CUDA device (#7869) 
* Raise error when using AMP on non-CUDA device

* make style

* make style
 2020-10-19 15:59:30 -04:00
Patrick von Platen
e3d2bee8d0
fix t5 training docstring (#7911) 2020-10-19 21:49:47 +02:00
Ayub Subhaniya
df1ddcedf2
decoder_config used before intialisation (#7903) 
Seeing error when sending `decoder_config` as a parameter while initializing a encoder-decoder model from pretrained. 
fixed "UnboundLocalError: local variable 'decoder_config' referenced before assignment"
 2020-10-19 19:48:49 +02:00
Quentin Lhoest
033f29c625
Allow Custom Dataset in RAG Retriever (#7763) 
* add CustomHFIndex

* typo in config

* update tests

* add custom dataset example

* clean script

* update test data

* minor in test

* docs

* docs

* style

* fix imports

* allow to pass the indexed dataset directly

* update tests

* use multiset DPR

* address thom and patrick's comments

* style

* update dpr tokenizer

* add output_dir flag in use_own_knowledge_dataset.py

* allow custom datasets in examples/rag/finetune.py

* add test for custom dataset in distributed rag retriever
 2020-10-19 19:42:45 +02:00