Yossi Synett
bc0d26d1de
[All Seq2Seq model + CLM models that can be used with EncoderDecoder] Add cross-attention weights to outputs ( #8071 )
...
* Output cross-attention with decoder attention output
* Update src/transformers/modeling_bert.py
* add cross-attention for t5 and bart as well
* fix tests
* correct typo in docs
* add sylvains and sams comments
* correct typo
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2020-11-06 19:34:48 +01:00
Weizhen
2422cda01b
ProphetNet ( #7157 )
...
* add new model prophetnet
prophetnet modified
modify codes as suggested v1
add prophetnet test files
* still bugs, because of changed output formats of encoder and decoder
* move prophetnet into the latest version
* clean integration tests
* clean tokenizers
* add xlm config to init
* correct typo in init
* further refactoring
* continue refactor
* save parallel
* add decoder_attention_mask
* fix use_cache vs. past_key_values
* fix common tests
* change decoder output logits
* fix xlm tests
* make common tests pass
* change model architecture
* add tokenizer tests
* finalize model structure
* no weight mapping
* correct n-gram stream attention mask as discussed with qweizhen
* remove unused import
* fix index.rst
* fix tests
* delete unnecessary code
* add fast integration test
* rename weights
* final weight remapping
* save intermediate
* Descriptions for Prophetnet Config File
* finish all models
* finish new model outputs
* delete unnecessary files
* refactor encoder layer
* add dummy docs
* code quality
* fix tests
* add model pages to doctree
* further refactor
* more refactor, more tests
* finish code refactor and tests
* remove unnecessary files
* further clean up
* add docstring template
* finish tokenizer doc
* finish prophetnet
* fix copies
* fix typos
* fix tf tests
* fix fp16
* fix tf test 2nd try
* fix code quality
* add test for each model
* merge new tests to branch
* Update model_cards/microsoft/prophetnet-large-uncased-cnndm/README.md
Co-authored-by: Sam Shleifer <sshleifer@gmail.com>
* Update model_cards/microsoft/prophetnet-large-uncased-cnndm/README.md
Co-authored-by: Sam Shleifer <sshleifer@gmail.com>
* Update src/transformers/modeling_prophetnet.py
Co-authored-by: Sam Shleifer <sshleifer@gmail.com>
* Update utils/check_repo.py
Co-authored-by: Sam Shleifer <sshleifer@gmail.com>
* apply sams and sylvains comments
* make style
* remove unnecessary code
* Update README.md
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update README.md
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/configuration_prophetnet.py
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
* implement lysandres comments
* correct docs
* fix isort
* fix tokenizers
* fix copies
Co-authored-by: weizhen <weizhen@mail.ustc.edu.cn>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Sam Shleifer <sshleifer@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
2020-10-19 17:36:09 +02:00
Patrick von Platen
221d4c63a3
clean naming ( #7068 )
2020-09-11 09:57:53 +02:00
Patrick von Platen
7fd1febf38
Add "Leveraging Pretrained Checkpoints for Generation" Seq2Seq models. ( #6594 )
...
* add conversion script
* improve conversion script
* make style
* add tryout files
* fix
* update
* add causal bert
* better names
* add tokenizer file as well
* finish causal_bert
* fix small bugs
* improve generate
* change naming
* renaming
* renaming
* renaming
* remove leftover files
* clean files
* add fix tokenizer
* finalize
* correct slow test
* update docs
* small fixes
* fix link
* adapt check repo
* apply sams and sylvains recommendations
* fix import
* implement Lysandres recommendations
* fix logger warn
2020-09-10 16:40:51 +02:00
Patrick von Platen
afc4ece462
[Generate] Facilitate PyTorch generate using ModelOutputs ( #6735 )
...
* fix generate for GPT2 Double Head
* fix gpt2 double head model
* fix bart / t5
* also add for no beam search
* fix no beam search
* fix encoder decoder
* simplify t5
* simplify t5
* fix t5 tests
* fix BART
* fix transfo-xl
* fix conflict
* integrating sylvains and sams comments
* fix tf past_decoder_key_values
* fix enc dec test
2020-09-01 12:38:25 +02:00
Lysandre
a75c64d80c
Black 20 release
2020-08-26 17:20:22 +02:00
Sylvain Gugger
a573777901
Update repo to isort v5 ( #6686 )
...
* Run new isort
* More changes
* Update CI, CONTRIBUTING and benchmarks
2020-08-24 11:03:01 -04:00
Patrick von Platen
fe0b85e77a
[EncoderDecoder] Add functionality to tie encoder decoder weights ( #6538 )
...
* start adding tie encoder to decoder functionality
* finish model tying
* make style
* Apply suggestions from code review
* fix t5 list including cross attention
* apply sams suggestions
* Update src/transformers/modeling_encoder_decoder.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* add max depth break point
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2020-08-19 14:23:45 +02:00
Patrick von Platen
1d6e71e116
[EncoderDecoder] Add Cross Attention for GPT2 ( #6415 )
...
* add cross attention layers for gpt2
* make gpt2 cross attention work
* finish bert2gpt2
* add explicit comments
* remove attention mask since not yet supported
* revert attn mask in pipeline
* Update src/transformers/modeling_gpt2.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/modeling_encoder_decoder.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2020-08-14 09:43:29 +02:00
Patrick von Platen
0735def8e1
[EncoderDecoder] Add encoder-decoder for roberta/ vanilla longformer ( #6411 )
...
* add encoder-decoder for roberta
* fix headmask
* apply Sylvains suggestions
* fix typo
* Apply suggestions from code review
2020-08-12 18:23:30 +02:00
Patrick von Platen
3425936643
[EncoderDecoderModel] add a add_cross_attention boolean to config ( #6377 )
...
* correct encoder decoder model
* Apply suggestions from code review
* apply sylvains suggestions
2020-08-10 19:46:48 +02:00
Stas Bekman
5deed37f9f
cleanup torch unittests ( #6196 )
...
* improve unit tests
this is a sample of one test according to the request in https://github.com/huggingface/transformers/issues/5973
before I apply it to the rest
* batch 1
* batch 2
* batch 3
* batch 4
* batch 5
* style
* non-tf template
* last deletion of check_loss_output
2020-08-04 02:42:56 -04:00
Sam Shleifer
13deb95a40
Move tests/utils.py -> transformers/testing_utils.py ( #5350 )
2020-07-01 10:31:17 -04:00
Sylvain Gugger
1e2631d6f8
Split LMBert model in two ( #4874 )
...
* Split LMBert model in two
* Fix example
* Remove lm_labels
* Adapt tests, refactor prepare_for_generation
* Fix merge
* Hide BeartLMHeadModel
2020-06-10 18:26:42 -04:00
Patrick von Platen
8cca875569
[EncoderDecoderConfig] automatically set decoder config to decoder ( #4809 )
...
* automatically set decoder config to decoder
* add more tests
2020-06-05 23:16:37 +02:00
Sylvain Gugger
1b5820a565
Unify label args ( #4722 )
...
* Deprecate masked_lm_labels argument
* Apply to all models
* Better error message
2020-06-03 09:36:26 -04:00
Patrick von Platen
0866669e75
[EncoderDecoder] Fix initialization and save/load bug ( #4680 )
...
* fix bug
* add more tests
2020-05-30 01:25:19 +02:00
Patrick von Platen
aa925a52fa
[Tests, GPU, SLOW] fix a bunch of GPU hardcoded tests in Pytorch ( #4468 )
...
* fix gpu slow tests in pytorch
* change model to device syntax
2020-05-19 21:35:04 +02:00
Patrick von Platen
8e67573a64
[EncoderDecoder Tests] Improve tests ( #4046 )
...
* Hoist bert model tester for patric
* indent
* make tests work
* Update tests/test_modeling_bert.py
Co-authored-by: Julien Chaumond <chaumond@gmail.com>
Co-authored-by: sshleifer <sshleifer@gmail.com>
Co-authored-by: Julien Chaumond <chaumond@gmail.com>
2020-05-04 02:18:36 +02:00
Patrick von Platen
fa49b9afea
Clean Encoder-Decoder models with Bart/T5-like API and add generate possibility ( #3383 )
...
* change encoder decoder style to bart & t5 style
* make encoder decoder generation dummy work for bert
* make style
* clean init config in encoder decoder
* add tests for encoder decoder models
* refactor and add last tests
* refactor and add last tests
* fix attn masks for bert encoder decoder
* make style
* refactor prepare inputs for Bert
* refactor
* finish encoder decoder
* correct typo
* add docstring to config
* finish
* add tests
* better naming
* make style
* fix flake8
* clean docstring
* make style
* rename
2020-04-28 15:11:09 +02:00
Sam Shleifer
9df74b8bc4
Delete all mentions of Model2Model ( #3019 )
2020-02-26 11:36:27 -05:00
alberduris
81d6841b4b
GPU text generation: mMoved the encoded_prompt to correct device
2020-01-06 15:11:12 +01:00
alberduris
dd4df80f0b
Moved the encoded_prompts to correct device
2020-01-06 15:11:12 +01:00
Aymeric Augustin
7e98e211f0
Remove unittest.main() in test modules.
...
This construct isn't used anymore these days.
Running python tests/test_foo.py puts the tests/ directory on
PYTHONPATH, which isn't representative of how we run tests.
Use python -m unittest tests/test_foo.py instead.
2019-12-22 14:42:03 +01:00
Aymeric Augustin
ced0a94204
Switch test files to the standard test_*.py scheme.
2019-12-22 14:15:13 +01:00
