riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-31 02:02:21 +06:00
Code Issues Actions 23 Packages Projects Releases Wiki Activity
2,491 Commits 887 Branches 232 Tags 2.1 GiB
7bd11dda6f
Commit Graph

2491 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Lysandre
7bd11dda6f Release: v2.2.2 2019-12-13 16:45:30 -05:00
LysandreJik
c3248cf122 Tests for all tokenizers 2019-12-13 16:41:44 -05:00
Pascal Voitot
f2ac50cb55 better for python2.x 2019-12-13 16:41:44 -05:00
Pascal Voitot
4cbdc7d910 missed space 2019-12-13 16:41:44 -05:00
Pascal Voitot
dd2add9f6e more tests 2019-12-13 16:41:44 -05:00
Pascal Voitot
df160af736 🐛 #2096 in tokenizer.decode, space is not joined between all subtexts instead of before added tokens 2019-12-13 16:41:44 -05:00
Pascal Voitot
5b7b78e088 🐛 #2096 in tokenizer.decode, adds a space after special tokens to return right formatted string 2019-12-13 16:41:44 -05:00
Julien Chaumond
866d73ca26 [cli] Upload is now compatible with folders 2019-12-13 16:39:08 -05:00
Lysandre
d461472948 return for SQuAD [BLACKED] 2019-12-13 15:31:52 -05:00
Lysandre
f24a228a93 Speed up tokenization process 2019-12-13 14:50:35 -05:00
Lysandre
c8ed1c82c8 [SQUAD] Load checkpoint when evaluating without training 2019-12-13 12:13:48 -05:00
Pierric Cistac
5a5c4349e8
Fix summarization to_cpu doc 2019-12-13 10:02:33 -05:00
LysandreJik
7296f1010b Cleanup squad and add allow train_file and predict_file usage 2019-12-12 13:01:04 -05:00
Julien Chaumond
5d67aa21ae [doc] Replicate doc from #2144 2019-12-12 12:39:41 -05:00
LysandreJik
fe92755b99 Fix special tokens mask in encode 2019-12-12 11:37:19 -05:00
Alan deLevie
fbf5455a86 Fix typo in examples/run_glue.py args declaration. 
deay -> decay
 2019-12-12 11:16:19 -05:00
Thomas Wolf
90df44f0aa
Merge pull request #2063 from guillaume-be/special_tokens_mask_value_not_used 
special_tokens_mask value was unused and calculated twice
 2019-12-12 08:21:46 +01:00
Thomas Wolf
707f9e9241
Merge pull request #2081 from pglock/patch-1 
handle string with only whitespaces as empty
 2019-12-12 08:20:43 +01:00
Thomas Wolf
137e20a846
Merge pull request #2075 from huggingface/check-link-validity 
Check link validity
 2019-12-12 08:09:12 +01:00
Thomas Wolf
d5712f7cac
Merge branch 'master' into check-link-validity 2019-12-12 08:00:51 +01:00
Thomas Wolf
9c58b236ef
Merge pull request #2144 from huggingface/from-pretrained-from-url 
Allowing from_pretrained to load from url directly
 2019-12-12 07:43:40 +01:00
thomwolf
413f41921b fix merge 2019-12-12 07:34:42 +01:00
Thomas Wolf
386a93f0f8
Merge branch 'master' into from-pretrained-from-url 2019-12-12 07:31:05 +01:00
Thomas Wolf
2d103546ef
Merge pull request #2148 from huggingface/fix_encode_plus 
Fix encode plus
 2019-12-12 07:24:47 +01:00
Julien Chaumond
1748fdf657 [doc] Fix rst table 2019-12-11 18:32:27 -05:00
Julien Chaumond
36fc52a3b4 Update links to weights 2019-12-11 18:32:27 -05:00
Julien Chaumond
371c5ddfad Py2 tests for Lysandre 2019-12-11 18:32:27 -05:00
Julien Chaumond
5505cf7014 Run tests on Py2 too, for Lysandre 2019-12-11 18:32:27 -05:00
Julien Chaumond
9cb97c0c0f Actually run the tests 2019-12-11 18:32:27 -05:00
Julien Chaumond
95854c4a2f Actually run the tests 2019-12-11 18:32:27 -05:00
Julien Chaumond
d2100428d3 Update to new test infra and only run conditionally 2019-12-11 18:32:27 -05:00
Masatoshi Suzuki
597ba7feb3 Support testing Japanese BERT tokenizers 2019-12-11 18:32:27 -05:00
Masatoshi Suzuki
6a43dc9d7d Support Python 2 2019-12-11 18:32:27 -05:00
Masatoshi Suzuki
a09da4eeb0 Add a test for Japanese BERT tokenizers 2019-12-11 18:32:27 -05:00
Masatoshi Suzuki
57b5cb3eaa Fix loading BertJapaneseTokenizer 2019-12-11 18:32:27 -05:00
Masatoshi Suzuki
c03c0dfd23 Add support for Japanese BERT models by cl-tohoku 2019-12-11 18:32:27 -05:00
Julien Chaumond
4f15e5a267 Add tests. 
Maybe not the best possible place for the tests, lmk.
 2019-12-11 17:41:51 -05:00
Julien Chaumond
18e1f751f1 TF support 2019-12-11 17:07:46 -05:00
Julien Chaumond
31e5b5ff22 Fix tests + first example of doc 2019-12-11 15:22:02 -05:00
LysandreJik
3d57c51111 Fix encode plus 2019-12-11 15:10:17 -05:00
Julien Chaumond
c999a3e505 Allow from_pretrained to take a remote identifier 2019-12-11 12:29:58 -05:00
Stefan Schweter
030faccb8d doc: fix pretrained models table 2019-12-11 12:19:21 -05:00
thomwolf
29570db25b allowing from_pretrained to load from url directly 2019-12-11 17:19:18 +01:00
Julien Chaumond
2e2f9fed55
rm duplicate imports 2019-12-11 11:11:56 -05:00
LysandreJik
4c12860f7a Remove misleading documentation 2019-12-11 09:22:37 -05:00
Thomas Wolf
51ae203290
Merge pull request #2129 from leopd/master 
Progress indicator improvements when downloading pre-trained models.
 2019-12-10 22:18:55 +01:00
Leo Dirac
58d75aa310 Progress indicator improvements when downloading pre-trained models. 2019-12-10 11:36:56 -08:00
LysandreJik
6a73382706 Complete warning + cleanup 2019-12-10 14:33:24 -05:00
Lysandre
dc4e9e5cb3 DataParallel for SQuAD + fix XLM 2019-12-10 19:21:20 +00:00
Thomas Wolf
e6cff60b4c
Merge pull request #2069 from huggingface/cleaner-pt-tf-conversion 
clean up PT <=> TF conversion
 2019-12-10 15:34:08 +01:00