Patrick von Platen
7fd1febf38
Add "Leveraging Pretrained Checkpoints for Generation" Seq2Seq models. ( #6594 )
...
* add conversion script
* improve conversion script
* make style
* add tryout files
* fix
* update
* add causal bert
* better names
* add tokenizer file as well
* finish causal_bert
* fix small bugs
* improve generate
* change naming
* renaming
* renaming
* renaming
* remove leftover files
* clean files
* add fix tokenizer
* finalize
* correct slow test
* update docs
* small fixes
* fix link
* adapt check repo
* apply sams and sylvains recommendations
* fix import
* implement Lysandres recommendations
* fix logger warn
2020-09-10 16:40:51 +02:00
Patrick von Platen
afc4ece462
[Generate] Facilitate PyTorch generate using ModelOutputs ( #6735 )
...
* fix generate for GPT2 Double Head
* fix gpt2 double head model
* fix bart / t5
* also add for no beam search
* fix no beam search
* fix encoder decoder
* simplify t5
* simplify t5
* fix t5 tests
* fix BART
* fix transfo-xl
* fix conflict
* integrating sylvains and sams comments
* fix tf past_decoder_key_values
* fix enc dec test
2020-09-01 12:38:25 +02:00
Lysandre
a75c64d80c
Black 20 release
2020-08-26 17:20:22 +02:00
Sylvain Gugger
a573777901
Update repo to isort v5 ( #6686 )
...
* Run new isort
* More changes
* Update CI, CONTRIBUTING and benchmarks
2020-08-24 11:03:01 -04:00
Patrick von Platen
fe0b85e77a
[EncoderDecoder] Add functionality to tie encoder decoder weights ( #6538 )
...
* start adding tie encoder to decoder functionality
* finish model tying
* make style
* Apply suggestions from code review
* fix t5 list including cross attention
* apply sams suggestions
* Update src/transformers/modeling_encoder_decoder.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* add max depth break point
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2020-08-19 14:23:45 +02:00
Patrick von Platen
1d6e71e116
[EncoderDecoder] Add Cross Attention for GPT2 ( #6415 )
...
* add cross attention layers for gpt2
* make gpt2 cross attention work
* finish bert2gpt2
* add explicit comments
* remove attention mask since not yet supported
* revert attn mask in pipeline
* Update src/transformers/modeling_gpt2.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/modeling_encoder_decoder.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2020-08-14 09:43:29 +02:00
Patrick von Platen
0735def8e1
[EncoderDecoder] Add encoder-decoder for roberta/ vanilla longformer ( #6411 )
...
* add encoder-decoder for roberta
* fix headmask
* apply Sylvains suggestions
* fix typo
* Apply suggestions from code review
2020-08-12 18:23:30 +02:00
Patrick von Platen
3425936643
[EncoderDecoderModel] add a add_cross_attention boolean to config ( #6377 )
...
* correct encoder decoder model
* Apply suggestions from code review
* apply sylvains suggestions
2020-08-10 19:46:48 +02:00
Stas Bekman
5deed37f9f
cleanup torch unittests ( #6196 )
...
* improve unit tests
this is a sample of one test according to the request in https://github.com/huggingface/transformers/issues/5973
before I apply it to the rest
* batch 1
* batch 2
* batch 3
* batch 4
* batch 5
* style
* non-tf template
* last deletion of check_loss_output
2020-08-04 02:42:56 -04:00
Sam Shleifer
13deb95a40
Move tests/utils.py -> transformers/testing_utils.py ( #5350 )
2020-07-01 10:31:17 -04:00
Sylvain Gugger
1e2631d6f8
Split LMBert model in two ( #4874 )
...
* Split LMBert model in two
* Fix example
* Remove lm_labels
* Adapt tests, refactor prepare_for_generation
* Fix merge
* Hide BeartLMHeadModel
2020-06-10 18:26:42 -04:00
Patrick von Platen
8cca875569
[EncoderDecoderConfig] automatically set decoder config to decoder ( #4809 )
...
* automatically set decoder config to decoder
* add more tests
2020-06-05 23:16:37 +02:00
Sylvain Gugger
1b5820a565
Unify label args ( #4722 )
...
* Deprecate masked_lm_labels argument
* Apply to all models
* Better error message
2020-06-03 09:36:26 -04:00
Patrick von Platen
0866669e75
[EncoderDecoder] Fix initialization and save/load bug ( #4680 )
...
* fix bug
* add more tests
2020-05-30 01:25:19 +02:00
Patrick von Platen
aa925a52fa
[Tests, GPU, SLOW] fix a bunch of GPU hardcoded tests in Pytorch ( #4468 )
...
* fix gpu slow tests in pytorch
* change model to device syntax
2020-05-19 21:35:04 +02:00
Patrick von Platen
8e67573a64
[EncoderDecoder Tests] Improve tests ( #4046 )
...
* Hoist bert model tester for patric
* indent
* make tests work
* Update tests/test_modeling_bert.py
Co-authored-by: Julien Chaumond <chaumond@gmail.com>
Co-authored-by: sshleifer <sshleifer@gmail.com>
Co-authored-by: Julien Chaumond <chaumond@gmail.com>
2020-05-04 02:18:36 +02:00
Patrick von Platen
fa49b9afea
Clean Encoder-Decoder models with Bart/T5-like API and add generate possibility ( #3383 )
...
* change encoder decoder style to bart & t5 style
* make encoder decoder generation dummy work for bert
* make style
* clean init config in encoder decoder
* add tests for encoder decoder models
* refactor and add last tests
* refactor and add last tests
* fix attn masks for bert encoder decoder
* make style
* refactor prepare inputs for Bert
* refactor
* finish encoder decoder
* correct typo
* add docstring to config
* finish
* add tests
* better naming
* make style
* fix flake8
* clean docstring
* make style
* rename
2020-04-28 15:11:09 +02:00
Sam Shleifer
9df74b8bc4
Delete all mentions of Model2Model ( #3019 )
2020-02-26 11:36:27 -05:00
alberduris
81d6841b4b
GPU text generation: mMoved the encoded_prompt to correct device
2020-01-06 15:11:12 +01:00
alberduris
dd4df80f0b
Moved the encoded_prompts to correct device
2020-01-06 15:11:12 +01:00
Aymeric Augustin
7e98e211f0
Remove unittest.main() in test modules.
...
This construct isn't used anymore these days.
Running python tests/test_foo.py puts the tests/ directory on
PYTHONPATH, which isn't representative of how we run tests.
Use python -m unittest tests/test_foo.py instead.
2019-12-22 14:42:03 +01:00
Aymeric Augustin
ced0a94204
Switch test files to the standard test_*.py scheme.
2019-12-22 14:15:13 +01:00
