Sylvain Gugger
c89bdfbe72
Reorganize repo ( #8580 )
...
* Put models in subfolders
* Styling
* Fix imports in tests
* More fixes in test imports
* Sneaky hidden imports
* Fix imports in doc files
* More sneaky imports
* Finish fixing tests
* Fix examples
* Fix path for copies
* More fixes for examples
* Fix dummy files
* More fixes for example
* More model import fixes
* Is this why you're unhappy GitHub?
* Fix imports in conver command
2020-11-16 21:43:42 -05:00
Sam Shleifer
7abc1d96d1
no warn ( #8329 )
2020-11-05 11:42:24 -05:00
Stas Bekman
5423f2a9d4
[testing] port test_trainer_distributed to distributed pytest + TestCasePlus enhancements ( #8107 )
...
* move the helper code into testing_utils
* port test_trainer_distributed to work with pytest
* improve docs
* simplify notes
* doc
* doc
* style
* doc
* further improvements
* torch might not be available
* real fix
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2020-10-28 11:51:32 -04:00
Stas Bekman
023f0f3708
[s2s trainer] tests to use distributed on multi-gpu machine ( #7965 )
2020-10-22 17:26:22 -04:00
Stas Bekman
8b38173398
[seq2seq testing] multigpu test run via subprocess ( #7281 )
...
Co-authored-by: Sam Shleifer <sshleifer@gmail.com>
2020-10-21 17:20:53 -04:00
Stas Bekman
2255c2c7a0
[seq2seq] get_git_info fails gracefully ( #7843 )
...
Co-authored-by: Sam Shleifer <sshleifer@gmail.com>
2020-10-16 00:22:43 -04:00
Suraj Patil
99cb924bfb
[s2s] add config params like Dropout in Seq2SeqTrainingArguments ( #7532 )
2020-10-04 12:42:30 -04:00
Suraj Patil
72d363d979
[examples/s2s] clean up finetune_trainer ( #7509 )
2020-10-01 12:19:29 -04:00
Sam Shleifer
48f23f92a8
[s2sTrainer] test + code cleanup ( #7467 )
2020-10-01 00:33:01 -04:00
Amanpreet Singh
c031d01023
Seq2SeqDataset: avoid passing src_lang everywhere ( #7470 )
...
Co-authored-by: Sam Shleifer <sshleifer@gmail.com>
2020-09-30 13:27:48 -04:00
Sam Shleifer
7296fea1d6
[s2s] rougeLSum expects \n between sentences ( #7410 )
...
Co-authored-by: Swetha Mandava <smandava@nvidia.com>
2020-09-27 16:27:19 -04:00
Sam Shleifer
d9d0f1140b
[s2s] distributed eval allows num_return_sequences > 1 ( #7254 )
2020-09-24 17:30:09 -04:00
Sam Shleifer
656c27c3a3
[s2s] save hostname with repo info ( #7301 )
...
* save hostname
2020-09-21 17:26:24 -04:00
Sam Shleifer
a5638b2b3a
[s2s] dynamic batch size with --max_tokens_per_batch ( #7030 )
2020-09-17 15:19:34 -04:00
Sam Shleifer
0203ad43bc
[s2s] distributed eval cleanup ( #7186 )
2020-09-16 15:38:37 -04:00
Stas Bekman
fdaf8ab349
[s2s run_eval] new features ( #7109 )
...
Co-authored-by: Sam Shleifer <sshleifer@gmail.com>
2020-09-16 13:59:57 -04:00
Sam Shleifer
33d479d2b2
[s2s] distributed eval in one command ( #7124 )
2020-09-14 15:57:56 -04:00
Sam Shleifer
de9e297964
[s2s] distributed eval cleanup ( #7110 )
2020-09-13 23:40:38 -04:00
Sam Shleifer
e7f8d2ab64
[s2s] two stage run_distributed_eval.py ( #7105 )
2020-09-13 17:28:18 -04:00
Sam Shleifer
77950c485a
[wip/s2s] DistributedSortishSampler ( #7056 )
2020-09-10 15:23:44 -04:00
Sam Shleifer
a4fc0c80b1
[s2s] run_eval.py parses generate_kwargs ( #6948 )
2020-09-04 14:19:31 -04:00
Sam Shleifer
9336086ab5
prepare_seq2seq_batch makes labels/ decoder_input_ids made later. ( #6654 )
...
* broken test
* batch parity
* tests pass
* boom boom
* boom boom
* split out bart tokenizer tests
* fix tests
* boom boom
* Fixed dataset bug
* Fix marian
* Undo extra
* Get marian working
* Fix t5 tok tests
* Test passing
* Cleanup
* better assert msg
* require torch
* Fix mbart tests
* undo extra decoder_attn_mask change
* Fix import
* pegasus tokenizer can ignore src_lang kwargs
* unused kwarg test cov
* boom boom
* add todo for pegasus issue
* cover one word translation edge case
* Cleanup
* doc
2020-08-28 11:15:17 -04:00
Lysandre
a75c64d80c
Black 20 release
2020-08-26 17:20:22 +02:00
Sam Shleifer
0344428f79
[s2s] round bleu, rouge to 4 digits ( #6704 )
2020-08-25 00:33:11 -04:00
Sam Shleifer
e92efcf728
Mult rouge by 100: standard units ( #6359 )
2020-08-13 12:15:54 -04:00
Sam Shleifer
be1520d3a3
rename prepare_translation_batch -> prepare_seq2seq_batch ( #6103 )
2020-08-11 15:57:07 -04:00
Suraj Patil
9bed355449
[s2s] fix label_smoothed_nll_loss ( #6344 )
2020-08-08 04:21:12 -04:00
Sam Shleifer
2804fff839
[s2s]Use prepare_translation_batch for Marian finetuning ( #6293 )
...
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2020-08-06 14:58:38 -04:00
Sam Shleifer
dafa296c95
[s2s] Delete useless method, log tokens_per_batch ( #6081 )
2020-07-28 11:24:23 -04:00
Sam Shleifer
3c7fbf35a6
MBART: support summarization tasks where max_src_len > max_tgt_len ( #6003 )
...
* MBART: support summarization tasks
* fix test
* Style
* add tokenizer test
2020-07-28 08:18:11 -04:00
Sam Shleifer
5b193b39b0
[examples/seq2seq]: add --label_smoothing option ( #5919 )
2020-07-21 16:51:39 -04:00
Sam Shleifer
09a2f40684
Seq2SeqDataset uses linecache to save memory by @Pradhy729 ( #5792 )
...
Co-authored-by: Pradhy729 <49659913+Pradhy729@users.noreply.github.com>
2020-07-18 13:57:33 -04:00
Sam Shleifer
e238e3d55a
[seq2seq] Don't copy self.source in sortishsampler ( #5818 )
2020-07-17 01:53:25 -04:00
Sam Shleifer
1a647abf0b
[fix] check code quality ( #5772 )
2020-07-15 14:59:38 -04:00
Sam Shleifer
d0486c8bc2
[cleanup] T5 test, warnings ( #5761 )
2020-07-15 08:23:22 -04:00
Sam Shleifer
353b8f1e7a
Add mbart-large-cc25, support translation finetuning ( #5129 )
...
improve unittests for finetuning, especially w.r.t testing frozen parameters
fix freeze_embeds for T5
add streamlit setup.cfg
2020-07-07 13:23:01 -04:00
Sam Shleifer
393b8dc09a
examples/seq2seq/run_eval.py fixes and docs ( #5322 )
2020-06-26 19:20:43 -04:00
Thomas Wolf
601d4d699c
[tokenizers] Updates data processors, docstring, examples and model cards to the new API ( #5308 )
...
* remove references to old API in docstring - update data processors
* style
* fix tests - better type checking error messages
* better type checking
* include awesome fix by @LysandreJik for #5310
* updated doc and examples
2020-06-26 19:48:14 +02:00
Sam Shleifer
40457bcebb
examples/seq2seq supports translation ( #5202 )
2020-06-24 23:58:11 -04:00
