LysandreJik
bf503158c5
Sentence -> Sequence. Removed output_mask from the special token addition methods.
2019-09-19 10:55:06 +02:00
LysandreJik
8cba057260
Doc + remove artefacts
2019-09-19 10:55:06 +02:00
LysandreJik
6393261e41
encode + encode_plus tests modified
2019-09-19 10:55:06 +02:00
LysandreJik
dcc9bb3252
Modified encode to return only lists. Added a more complete encode_plus method
2019-09-19 10:55:06 +02:00
LysandreJik
af23b626c8
Max encoding length + corresponding tests
2019-09-19 10:55:06 +02:00
LysandreJik
c4d4f3ec8c
Updated DistilBERT test to reflect the sequence encoding
2019-09-19 10:55:06 +02:00
LysandreJik
d572d7027b
Number of added tokens calculator
2019-09-19 10:55:06 +02:00
LysandreJik
de8e14b6c0
Added DistilBERT to run_squad script
2019-09-19 10:55:06 +02:00
LysandreJik
88368c2a16
Added DistilBERT to run_lm_finetuning
2019-09-19 10:55:06 +02:00
LysandreJik
2d8ec5a684
Changed warning to be more explicit
...
Co-authored by: julien_c <chaumond@gmail.com>
2019-09-19 10:55:06 +02:00
LysandreJik
75635072e1
Updated GLUE script to add DistilBERT. Cleaned up unused args in the utils file.
2019-09-19 10:55:06 +02:00
LysandreJik
92a9976e91
Distilbert sequence builder w/ mask
2019-09-19 10:55:06 +02:00
LysandreJik
59057abe52
typo
2019-09-19 10:55:06 +02:00
LysandreJik
bac332fec0
Updated the GLUE data processor. Corrections to RoBERTa and XLNet.
2019-09-19 10:55:06 +02:00
LysandreJik
c3df2136e1
Added binary masking tests
2019-09-19 10:55:06 +02:00
LysandreJik
e391d4735e
Tokenizers' encode function can output binary masks
2019-09-19 10:55:06 +02:00
Thomas Wolf
0d1dad6d53
Merge pull request #1004 from erenup/master
...
Refactoring old run_swag.py
2019-09-18 21:42:51 +02:00
erenup
8960988f35
fixed to find best dev acc
2019-09-19 01:10:05 +08:00
erenup
b57bfb5fa0
Merge pull request #3 from erenup/run_multiple_choice_merge
...
Run multiple choice merge
2019-09-18 21:45:04 +08:00
erenup
46ffc28329
Merge branch 'master' into run_multiple_choice_merge
...
# Please enter a commit message to explain why this merge is necessary,
# especially if it merges an updated upstream into a topic branch.
#
# Lines starting with '#' will be ignored, and an empty message aborts
# the commit.
2019-09-18 21:43:46 +08:00
erenup
15143fbad6
move run_multiple_choice.py and utils_multiple_choice.py to examples
2019-09-18 21:18:46 +08:00
erenup
3cd6289758
Merge remote-tracking branch 'huggingface/master' into run_multiple_choice_merge
...
# Conflicts:
# examples/contrib/run_swag.py
2019-09-18 21:16:59 +08:00
erenup
36362cf086
move schedule.step after optimizer.step
2019-09-18 21:13:40 +08:00
thomwolf
e768f2322a
update run_openai_gpt to fix #1264
2019-09-18 10:07:47 +02:00
thomwolf
8334993915
clean up examples - updated to new keyword inputs - #1246
2019-09-18 10:01:27 +02:00
Julien Chaumond
62760baf46
tiny fixes
2019-09-17 18:29:15 -04:00
thomwolf
45de034bf8
fix #1223
2019-09-17 10:25:06 +02:00
erenup
5a81e79e25
Merge pull request #2 from erenup/run_multiple_choice_add_doc
...
Run multiple choice add doc
2019-09-16 22:39:54 +08:00
erenup
5882c442e5
add example usage
2019-09-16 22:38:08 +08:00
erenup
a9debaca3d
fixed init_weight
2019-09-16 19:55:24 +08:00
thomwolf
c88f05163d
fix typo in XLM models
2019-09-16 13:42:20 +02:00
erenup
982f181aa7
Merge remote-tracking branch 'origin/master' into run_multiple_choice_add_doc
2019-09-16 19:12:00 +08:00
erenup
84b9d1c423
Merge remote-tracking branch 'huggingface/master'
...
# Conflicts:
# pytorch_transformers/__init__.py
2019-09-16 19:06:12 +08:00
erenup
603b470a3d
add warnning info
2019-09-16 18:53:37 +08:00
erenup
4812a5a767
add doc string
2019-09-16 11:50:18 +08:00
VictorSanh
32e1332acf
[distil] fix once for all general logger for scripts
2019-09-11 14:19:07 +00:00
Thomas Wolf
b62abe87c9
Merge pull request #1249 from ziliwang/master
...
fixed: hard coding for max and min number will out of range in fp16, which will cause nan.
2019-09-11 15:53:28 +02:00
LysandreJik
11ac4b9555
[CI] Symbolic link for documentation
2019-09-11 10:13:44 +02:00
Zili Wang
8bdee1cb73
fixed: hard coding for max and min number will out of range in fp16, which will cause nan.
2019-09-11 15:41:53 +08:00
ziliwang
7424b2848f
Merge pull request #1 from huggingface/master
...
merege from original repo
2019-09-11 11:02:23 +08:00
VictorSanh
364920e216
fix small bug/typo
2019-09-10 21:45:01 +00:00
Thomas Wolf
23c23f5399
Merge pull request #1229 from SKRohit/master
...
changes in evaluate function in run_lm_finetuning.py
2019-09-10 22:16:45 +02:00
Thomas Wolf
99a54ac51c
Merge pull request #1233 from searchivarius/master
...
Fix to prevent crashing on assert len(tokens_b)>=1
2019-09-10 22:15:47 +02:00
Thomas Wolf
439b37b474
Merge pull request #1241 from mattolson93/patch-1
...
Fixing typo in gpt2 for doc site's class link
2019-09-10 22:14:18 +02:00
mattolson93
f2cf6ce4a9
Fixing typo in gpt2 for doc site's class link
2019-09-10 09:12:01 -07:00
Thomas Wolf
2c177a87eb
Merge pull request #1228 from huggingface/head-masking-test
...
Trying to fix the head masking test
2019-09-10 11:55:27 +02:00
searchivarius
eab980fd68
Fix to prevent crashing on assert len(tokens_b)>=1
2019-09-09 19:58:08 -04:00
VictorSanh
a95ced6260
[Distillation] save last chkpt as pytorch_model.bin
2019-09-09 19:53:35 +00:00
Rohit Kumar Singh
4b082bd4d8
Merge pull request #1 from SKRohit/SKRohit-patch-1
...
changes in return statement of evaluate function
2019-09-09 19:59:27 +05:30
Rohit Kumar Singh
e5df36397b
changes in return statement of evaluate function
...
changed `results` to `result` and removed `results` dict defined previously
2019-09-09 19:55:57 +05:30
