2922 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Pascal Voitot	4cbdc7d910	missed space	2019-12-13 16:41:44 -05:00
Pascal Voitot	dd2add9f6e	more tests	2019-12-13 16:41:44 -05:00
Pascal Voitot	df160af736	🐛 #2096 in tokenizer.decode, space is not joined between all subtexts instead of before added tokens	2019-12-13 16:41:44 -05:00
Pascal Voitot	5b7b78e088	🐛 #2096 in tokenizer.decode, adds a space after special tokens to return right formatted string	2019-12-13 16:41:44 -05:00
Julien Chaumond	866d73ca26	[cli] Upload is now compatible with folders	2019-12-13 16:39:08 -05:00
Lysandre	d461472948	return for SQuAD [BLACKED]	2019-12-13 15:31:52 -05:00
Lysandre	f24a228a93	Speed up tokenization process	2019-12-13 14:50:35 -05:00
Lysandre	c8ed1c82c8	[SQUAD] Load checkpoint when evaluating without training	2019-12-13 12:13:48 -05:00
thomwolf	5c00e344c1	update model doc - swith 3B/11B to 3b/11b	2019-12-13 16:33:29 +01:00
Morgan Funtowicz	0b51532ce9	Reintroducing the batch_encode_plus method	2019-12-13 16:22:50 +01:00
Thomas Wolf	110394b2ba	Merge branch 'master' into t5	2019-12-13 16:03:32 +01:00
Pierric Cistac	5a5c4349e8	Fix summarization to_cpu doc	2019-12-13 10:02:33 -05:00
thomwolf	8ade204098	fix tf	2019-12-13 14:48:47 +01:00
thomwolf	47f0e3cfb7	cleaning up configuration classes	2019-12-13 14:33:24 +01:00
Morgan Funtowicz	8938b546bf	Removed from_config	2019-12-13 14:27:04 +01:00
Morgan Funtowicz	1ca52567a4	Allow model conversion in the pipeline allocator.	2019-12-13 14:13:14 +01:00
Morgan Funtowicz	28e64ad5a4	Raise an exception if the pipeline allocator can't determine the tokenizer from the model.	2019-12-13 14:12:54 +01:00
Morgan Funtowicz	be5bf7b81b	Added NER pipeline.	2019-12-13 14:12:17 +01:00
Morgan Funtowicz	80eacb8f16	Adding labels mapping for classification models in their respective config.	2019-12-13 14:10:22 +01:00
thomwolf	33e72b08d5	fix inner dimensions for 3B/11B models	2019-12-13 11:33:05 +01:00
erenup	9b312f9d41	initial version for roberta squad	2019-12-13 14:51:40 +08:00
erenup	40ed717232	Merge remote-tracking branch 'refs/remotes/huggingface/master'	2019-12-13 09:10:17 +08:00
LysandreJik	7296f1010b	Cleanup squad and add allow train_file and predict_file usage	2019-12-12 13:01:04 -05:00
Julien Chaumond	5d67aa21ae	[doc] Replicate doc from #2144	2019-12-12 12:39:41 -05:00
LysandreJik	3fd71c4431	Update example scripts	2019-12-12 12:08:54 -05:00
LysandreJik	fe92755b99	Fix special tokens mask in encode	2019-12-12 11:37:19 -05:00
Alan deLevie	fbf5455a86	Fix typo in examples/run_glue.py args declaration. ... deay -> decay	2019-12-12 11:16:19 -05:00
thomwolf	f19dad61c7	fixing XLM conversion tests with dummy input	2019-12-12 14:46:30 +01:00
Morgan Funtowicz	f69dbecc38	Expose classification labels mapping (and reverse) in model config.	2019-12-12 10:25:36 +01:00
Thomas Wolf	90df44f0aa	Merge pull request #2063 from guillaume-be/special_tokens_mask_value_not_used ... special_tokens_mask value was unused and calculated twice	2019-12-12 08:21:46 +01:00
Thomas Wolf	707f9e9241	Merge pull request #2081 from pglock/patch-1 ... handle string with only whitespaces as empty	2019-12-12 08:20:43 +01:00
Thomas Wolf	137e20a846	Merge pull request #2075 from huggingface/check-link-validity ... Check link validity	2019-12-12 08:09:12 +01:00
Thomas Wolf	d5712f7cac	Merge branch 'master' into check-link-validity	2019-12-12 08:00:51 +01:00
Thomas Wolf	9c58b236ef	Merge pull request #2144 from huggingface/from-pretrained-from-url ... Allowing from_pretrained to load from url directly	2019-12-12 07:43:40 +01:00
thomwolf	413f41921b	fix merge	2019-12-12 07:34:42 +01:00
Thomas Wolf	386a93f0f8	Merge branch 'master' into from-pretrained-from-url	2019-12-12 07:31:05 +01:00
Thomas Wolf	2d103546ef	Merge pull request #2148 from huggingface/fix_encode_plus ... Fix encode plus	2019-12-12 07:24:47 +01:00
Julien Chaumond	1748fdf657	[doc] Fix rst table	2019-12-11 18:32:27 -05:00
Julien Chaumond	36fc52a3b4	Update links to weights	2019-12-11 18:32:27 -05:00
Julien Chaumond	371c5ddfad	Py2 tests for Lysandre	2019-12-11 18:32:27 -05:00
Julien Chaumond	5505cf7014	Run tests on Py2 too, for Lysandre	2019-12-11 18:32:27 -05:00
Julien Chaumond	9cb97c0c0f	Actually run the tests	2019-12-11 18:32:27 -05:00
Julien Chaumond	95854c4a2f	Actually run the tests	2019-12-11 18:32:27 -05:00
Julien Chaumond	d2100428d3	Update to new test infra and only run conditionally	2019-12-11 18:32:27 -05:00
Masatoshi Suzuki	597ba7feb3	Support testing Japanese BERT tokenizers	2019-12-11 18:32:27 -05:00
Masatoshi Suzuki	6a43dc9d7d	Support Python 2	2019-12-11 18:32:27 -05:00
Masatoshi Suzuki	a09da4eeb0	Add a test for Japanese BERT tokenizers	2019-12-11 18:32:27 -05:00
Masatoshi Suzuki	57b5cb3eaa	Fix loading BertJapaneseTokenizer	2019-12-11 18:32:27 -05:00
Masatoshi Suzuki	c03c0dfd23	Add support for Japanese BERT models by cl-tohoku	2019-12-11 18:32:27 -05:00
Julien Chaumond	4f15e5a267	Add tests. ... Maybe not the best possible place for the tests, lmk.	2019-12-11 17:41:51 -05:00