Commit Graph

19383 Commits

Author SHA1 Message Date
thomwolf
b514a60c36 added tests for OpenAI GPT and Transformer-XL tokenizers 2019-02-11 10:17:16 +01:00
thomwolf
9bdcba53fd fix tests 2019-02-09 17:07:12 +01:00
thomwolf
f0bf81e141 back compatibility with Path inputs in fle_utils 2019-02-09 17:05:23 +01:00
thomwolf
9f9909ea2f update readme 2019-02-09 16:59:21 +01:00
thomwolf
6cd769957e update transfo xl example 2019-02-09 16:59:17 +01:00
thomwolf
1320e4ec0c mc_token_mask => mc_token_ids 2019-02-09 16:58:53 +01:00
thomwolf
f4a07a392c mems not splitted 2019-02-09 16:14:31 +01:00
thomwolf
43b9af0cac mems initialized to None in run_transfo 2019-02-09 16:12:19 +01:00
thomwolf
cfcb95417c fix hasattr 2019-02-08 23:08:53 +01:00
thomwolf
0c1a6f9b1d update readme 2019-02-08 22:32:25 +01:00
thomwolf
1756b5e956 fix loading from Transfo-XL LM model 2019-02-08 22:32:17 +01:00
thomwolf
dadd0c1b13 updating __main__ 2019-02-08 22:31:57 +01:00
thomwolf
102c6b238c adding file cache to __init__ 2019-02-08 22:31:46 +01:00
thomwolf
b80684b23f fixing run openai gpt example 2019-02-08 22:31:32 +01:00
thomwolf
80607874c1 fix layer norm epsilon in OpenAI GPT 2019-02-08 21:49:05 +01:00
thomwolf
7b4b0cf966 logging 2019-02-08 11:16:29 +01:00
thomwolf
4bbb9f2d68 log loss - helpers 2019-02-08 11:14:29 +01:00
thomwolf
5d7e845712 fix model on cuda 2019-02-08 11:08:43 +01:00
thomwolf
eccb2f0163 hot fix 2019-02-08 11:05:20 +01:00
thomwolf
5adc20723b add distant debugging 2019-02-08 11:03:59 +01:00
thomwolf
5ee4f17234 adding option to load on cpu 2019-02-08 10:37:40 +01:00
Thomas Wolf
2dfaf2f227
Merge pull request #261 from deepset-ai/rm_arg_lm_finetuning
removing unused argument eval_batch_size from LM finetuning #256
2019-02-08 10:36:03 +01:00
thomwolf
777459b471 run openai example running 2019-02-08 10:33:14 +01:00
thomwolf
edcb56fd96 more explicit variable name 2019-02-08 09:54:49 +01:00
thomwolf
6bc082da0a updating examples 2019-02-08 00:02:26 +01:00
thomwolf
eb8fda51f4 update docstrings 2019-02-07 23:15:20 +01:00
thomwolf
e77721e4fe renamed examples 2019-02-07 23:15:15 +01:00
thomwolf
009b581316 updated readme 2019-02-07 23:15:05 +01:00
thomwolf
f99f2fb661 docstrings 2019-02-07 17:07:22 +01:00
thomwolf
438db43d46 update adaptive softmax head 2019-02-07 17:07:15 +01:00
thomwolf
c306869ea2 add two transformer xl models 2019-02-07 17:07:03 +01:00
thomwolf
d482e3d79d adding examples for openai and transformer-xl 2019-02-07 17:06:41 +01:00
thomwolf
9c3c24800b split saved model in config & weights 2019-02-07 17:06:17 +01:00
thomwolf
2df41663f1 added test 2019-02-07 17:05:49 +01:00
tholor
9aebc711c9 adjust error message related to args.do_eval 2019-02-07 11:49:38 +01:00
tholor
4a450b25d5 removing unused argument eval_batch_size from LM finetuning #256 2019-02-07 10:06:38 +01:00
Thomas Wolf
58f0a2745c
Merge pull request #258 from BoeingX/master
Fix the undefined variable in squad example
2019-02-06 20:33:18 +01:00
Baoyang Song
7ac3311e48
Fix the undefined variable in squad example 2019-02-06 19:36:08 +01:00
thomwolf
ed47cb6cba fixing transfo eval script 2019-02-06 16:22:17 +01:00
thomwolf
973926431e fix differencies with tensorflow version (mem cells and adaptive sofmax clusters) 2019-02-06 15:42:29 +01:00
thomwolf
ba9e4eb354 fix unicode in tokenization tests 2019-02-06 00:28:00 +01:00
thomwolf
34bdb7f9cb update circle-ci for python 2.7 and 3.5 2019-02-06 00:25:12 +01:00
Thomas Wolf
848aae49e1
Merge branch 'master' into python_2 2019-02-06 00:13:20 +01:00
thomwolf
448937c00d python 2 compatibility 2019-02-06 00:07:46 +01:00
thomwolf
ba37ddc5ce fix run_lm_modeling example command line 2019-02-06 00:07:08 +01:00
thomwolf
822915142b fix docstring 2019-02-05 16:34:32 +01:00
Thomas Wolf
bd74632687
Merge pull request #251 from Iwontbecreative/active_loss_tok_classif
Only keep the active part mof the loss for token classification
2019-02-05 16:33:45 +01:00
Thomas Wolf
fd223374f0
Merge pull request #208 from Liangtaiwan/mergesquad
Merge run_squad.py and run_squad2.py
2019-02-05 16:15:03 +01:00
thomwolf
d609ba24cb resolving merge conflicts 2019-02-05 16:14:25 +01:00
thomwolf
bde1eeebe0 rename 2019-02-05 16:11:22 +01:00