riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-31 02:02:21 +06:00
Code Issues Actions 23 Packages Projects Releases Wiki Activity
1,455 Commits 887 Branches 232 Tags 2.1 GiB
09363f2a8b
Commit Graph

1455 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
LysandreJik
09363f2a8b Fix documentation index 2019-08-30 19:48:32 -04:00
Thomas Wolf
51e980ce36
Merge pull request #1155 from anhnt170489/apex_fp16 
Update apex fp16 implementation
 2019-08-30 23:29:11 +02:00
Thomas Wolf
206c35e9a4
Merge pull request #1154 from ziliwang/master 
fix: hard coding for max number
 2019-08-30 23:23:08 +02:00
Thomas Wolf
f3d18c71ec
Merge pull request #1152 from epwalsh/fix-special-tokens 
fix adding special tokens
 2019-08-30 23:21:59 +02:00
Thomas Wolf
d483cd8e46
Merge pull request #1074 from huggingface/improved_testing 
Shortcut to special tokens' ids - fix GPT2 & RoBERTa tokenizers - improved testing for GPT/GPT-2
 2019-08-30 23:18:58 +02:00
Thomas Wolf
d2f21f08f5
Merge pull request #1092 from shijie-wu/xlm-tokenization 
Added cleaned configuration properties for tokenizer with serialization - improve tokenization of XLM
 2019-08-30 23:15:40 +02:00
Thomas Wolf
12b9cc9e26
Merge pull request #1110 from huggingface/automodels 
Torch.hub now based on AutoModels - Updating AutoModels with AutoModelWithLMHead, Sequence Classification and Question Answering
 2019-08-30 23:08:57 +02:00
thomwolf
bfe93a5a21 fix distilbert in auto tokenizer 2019-08-30 22:43:26 +02:00
thomwolf
256086bc69 clean up and simplify hubconf 2019-08-30 22:34:23 +02:00
thomwolf
80aa87d9a3 fix distilbert tokenizer 2019-08-30 22:24:23 +02:00
thomwolf
455a4c842c add distilbert tokenizer 2019-08-30 22:20:51 +02:00
thomwolf
7a1f174a9d update names of torch.hub to simpler names - update docstring 2019-08-30 22:20:44 +02:00
thomwolf
c665e0fcfe Merge branch 'automodels' of https://github.com/huggingface/pytorch-transformers into automodels 2019-08-30 21:53:36 +02:00
LysandreJik
9b6e3b34d9 Docstrings 2019-08-30 14:09:02 -04:00
LysandreJik
dec8f4d6fd Added DistilBERT models to all other AutoModels. 2019-08-30 13:52:18 -04:00
LysandreJik
bc29aa67a9 HubConf configuration 2019-08-30 12:48:55 -04:00
thomwolf
f35f612280 updating docstring for AutoModel 2019-08-30 12:48:55 -04:00
LysandreJik
7ca9653852 Pytorch Hub & AutoModels 2019-08-30 12:48:55 -04:00
LysandreJik
25e8389439 Tests for added AutoModels 2019-08-30 12:48:55 -04:00
LysandreJik
dc43215c01 Added multiple AutoModel classes: AutoModelWithLMHead, AutoModelForQuestionAnswering and AutoModelForSequenceClassification 2019-08-30 12:48:55 -04:00
VictorSanh
282c276e09 typos + file name coherence in distillation README 2019-08-30 12:02:29 -04:00
VictorSanh
803c1cc4ea fix relative import bug cf Issue #1140 2019-08-30 12:01:27 -04:00
thomwolf
7044ed6b05 fix tokenizers serialization 2019-08-30 17:36:11 +02:00
Thomas Wolf
cd65c41a83
Merge branch 'master' into xlm-tokenization 2019-08-30 17:15:16 +02:00
thomwolf
69da972ace added test and debug tokenizer configuration serialization 2019-08-30 17:09:36 +02:00
thomwolf
88111de07c saving and reloading tokenizer configurations 2019-08-30 16:55:48 +02:00
Thomas Wolf
b66e9b4433
Merge pull request #1158 from rabeehk/master 
regarding #1026 pull request
 2019-08-30 16:30:33 +02:00
Thomas Wolf
0a2fecdf90
Merge branch 'master' into master 2019-08-30 16:30:08 +02:00
thomwolf
3871b8a107 adding xlm 17 and 100 models and config on aws 2019-08-30 16:28:42 +02:00
thomwolf
8678ff8df5 adding 17 and 100 xlm models 2019-08-30 16:26:04 +02:00
LysandreJik
e0caab0cf0 fix link 2019-08-30 10:09:17 -04:00
LysandreJik
a600b30cc3 Fix index number in documentation 2019-08-30 10:08:14 -04:00
LysandreJik
20c06fa37d Added DistilBERT to documentation index 2019-08-30 10:06:51 -04:00
Rabeeh KARIMI
39eb31e11e remove reloading tokenizer in the training, adding it to the evaluation part 2019-08-30 15:44:41 +02:00
Rabeeh KARIMI
350bb6bffa updated tokenizer loading for addressing reproducibility issues 2019-08-30 15:34:28 +02:00
thomwolf
82462c5cba Added option to setup pretrained tokenizer arguments 2019-08-30 15:30:41 +02:00
Thomas Wolf
41f35d0b3d
Merge pull request #1089 from dhpollack/dhp/use_pytorch_layernorm 
change layernorm code to pytorch's native layer norm
 2019-08-30 14:49:08 +02:00
Thomas Wolf
01ad55f8cf
Merge pull request #1026 from rabeehk/master 
loads the tokenizer for each checkpoint, to solve the reproducability…
 2019-08-30 14:15:36 +02:00
Thomas Wolf
50e615f43d
Merge branch 'master' into improved_testing 2019-08-30 13:40:35 +02:00
thomwolf
f8aace6bcd update tokenizers to use self.XX_token_id instead of converting self.XX_token 2019-08-30 13:39:52 +02:00
thomwolf
8faf2e086b more doc on special tokens 2019-08-30 13:36:22 +02:00
Thomas Wolf
f7978490b2
Merge pull request #1148 from huggingface/circleci 
Documentation auto-deploy
 2019-08-30 13:28:16 +02:00
thomwolf
ce5ef4b35d python2 doesn't spark joy 2019-08-30 13:22:43 +02:00
thomwolf
5dd7b677ad clean up all byte-level bpe tests 2019-08-30 12:43:08 +02:00
thomwolf
ca1a00a302 fix for python2 2019-08-30 12:29:31 +02:00
thomwolf
4e6a3172ce update roberta docstring as well 2019-08-30 12:23:37 +02:00
thomwolf
fd10d79b55 update GPT2 docstring 2019-08-30 12:23:12 +02:00
thomwolf
abe734ca1f fix GPT-2 and RoBERTa tests to be clean now 2019-08-30 12:20:18 +02:00
thomwolf
0f5a799456 fix GPT2DoubleHeadModel docstring 2019-08-30 11:49:23 +02:00
thomwolf
d51f72d5de adding shortcut to the ids of all the special tokens 2019-08-30 11:41:11 +02:00