Commit Graph

19383 Commits

Author SHA1 Message Date
Thomas Wolf
a52d56c8d9
Merge branch 'master' into cleanup-configs 2019-12-14 09:43:07 +01:00
Thomas Wolf
e92bcb7eb6
Merge pull request #1739 from huggingface/t5
[WIP] Adding Google T5 model
2019-12-14 09:40:43 +01:00
thomwolf
cbb368ca06 distilbert tests 2019-12-14 09:31:18 +01:00
Julien Chaumond
b6d4284b26 [cli] Uploads: fix + test edge case 2019-12-13 22:44:57 -05:00
erenup
a1faaf9962 deleted useless file 2019-12-14 08:57:13 +08:00
erenup
c7780700f5 Merge branch 'refs/heads/squad_roberta'
# Conflicts:
#	transformers/data/processors/squad.py
2019-12-14 08:53:59 +08:00
erenup
76f0d99f02 Merge remote-tracking branch 'refs/remotes/huggingface/master' 2019-12-14 08:45:17 +08:00
erenup
8e9526b4b5 add multiple processing 2019-12-14 08:43:58 +08:00
Lysandre
7bd11dda6f Release: v2.2.2 2019-12-13 16:45:30 -05:00
LysandreJik
c3248cf122 Tests for all tokenizers 2019-12-13 16:41:44 -05:00
Pascal Voitot
f2ac50cb55 better for python2.x 2019-12-13 16:41:44 -05:00
Pascal Voitot
4cbdc7d910 missed space 2019-12-13 16:41:44 -05:00
Pascal Voitot
dd2add9f6e more tests 2019-12-13 16:41:44 -05:00
Pascal Voitot
df160af736 🐛 #2096 in tokenizer.decode, space is not joined between all subtexts instead of before added tokens 2019-12-13 16:41:44 -05:00
Pascal Voitot
5b7b78e088 🐛 #2096 in tokenizer.decode, adds a space after special tokens to return right formatted string 2019-12-13 16:41:44 -05:00
Julien Chaumond
866d73ca26 [cli] Upload is now compatible with folders 2019-12-13 16:39:08 -05:00
Lysandre
d461472948 return for SQuAD [BLACKED] 2019-12-13 15:31:52 -05:00
Lysandre
f24a228a93 Speed up tokenization process 2019-12-13 14:50:35 -05:00
Lysandre
c8ed1c82c8 [SQUAD] Load checkpoint when evaluating without training 2019-12-13 12:13:48 -05:00
thomwolf
5c00e344c1 update model doc - swith 3B/11B to 3b/11b 2019-12-13 16:33:29 +01:00
Morgan Funtowicz
0b51532ce9 Reintroducing the batch_encode_plus method 2019-12-13 16:22:50 +01:00
Thomas Wolf
110394b2ba
Merge branch 'master' into t5 2019-12-13 16:03:32 +01:00
Pierric Cistac
5a5c4349e8
Fix summarization to_cpu doc 2019-12-13 10:02:33 -05:00
thomwolf
8ade204098 fix tf 2019-12-13 14:48:47 +01:00
thomwolf
47f0e3cfb7 cleaning up configuration classes 2019-12-13 14:33:24 +01:00
Morgan Funtowicz
8938b546bf Removed from_config 2019-12-13 14:27:04 +01:00
Morgan Funtowicz
1ca52567a4 Allow model conversion in the pipeline allocator. 2019-12-13 14:13:14 +01:00
Morgan Funtowicz
28e64ad5a4 Raise an exception if the pipeline allocator can't determine the tokenizer from the model. 2019-12-13 14:12:54 +01:00
Morgan Funtowicz
be5bf7b81b Added NER pipeline. 2019-12-13 14:12:17 +01:00
Morgan Funtowicz
80eacb8f16 Adding labels mapping for classification models in their respective config. 2019-12-13 14:10:22 +01:00
thomwolf
33e72b08d5 fix inner dimensions for 3B/11B models 2019-12-13 11:33:05 +01:00
erenup
9b312f9d41 initial version for roberta squad 2019-12-13 14:51:40 +08:00
erenup
40ed717232 Merge remote-tracking branch 'refs/remotes/huggingface/master' 2019-12-13 09:10:17 +08:00
LysandreJik
7296f1010b Cleanup squad and add allow train_file and predict_file usage 2019-12-12 13:01:04 -05:00
Julien Chaumond
5d67aa21ae [doc] Replicate doc from #2144 2019-12-12 12:39:41 -05:00
LysandreJik
3fd71c4431 Update example scripts 2019-12-12 12:08:54 -05:00
LysandreJik
fe92755b99 Fix special tokens mask in encode 2019-12-12 11:37:19 -05:00
Alan deLevie
fbf5455a86 Fix typo in examples/run_glue.py args declaration.
deay -> decay
2019-12-12 11:16:19 -05:00
thomwolf
f19dad61c7 fixing XLM conversion tests with dummy input 2019-12-12 14:46:30 +01:00
Morgan Funtowicz
f69dbecc38 Expose classification labels mapping (and reverse) in model config. 2019-12-12 10:25:36 +01:00
Thomas Wolf
90df44f0aa
Merge pull request #2063 from guillaume-be/special_tokens_mask_value_not_used
special_tokens_mask value was unused and calculated twice
2019-12-12 08:21:46 +01:00
Thomas Wolf
707f9e9241
Merge pull request #2081 from pglock/patch-1
handle string with only whitespaces as empty
2019-12-12 08:20:43 +01:00
Thomas Wolf
137e20a846
Merge pull request #2075 from huggingface/check-link-validity
Check link validity
2019-12-12 08:09:12 +01:00
Thomas Wolf
d5712f7cac
Merge branch 'master' into check-link-validity 2019-12-12 08:00:51 +01:00
Thomas Wolf
9c58b236ef
Merge pull request #2144 from huggingface/from-pretrained-from-url
Allowing from_pretrained to load from url directly
2019-12-12 07:43:40 +01:00
thomwolf
413f41921b fix merge 2019-12-12 07:34:42 +01:00
Thomas Wolf
386a93f0f8
Merge branch 'master' into from-pretrained-from-url 2019-12-12 07:31:05 +01:00
Thomas Wolf
2d103546ef
Merge pull request #2148 from huggingface/fix_encode_plus
Fix encode plus
2019-12-12 07:24:47 +01:00
Julien Chaumond
1748fdf657 [doc] Fix rst table 2019-12-11 18:32:27 -05:00
Julien Chaumond
36fc52a3b4 Update links to weights 2019-12-11 18:32:27 -05:00