552 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Catalin Voss	4a49c22584	Warn instead of raising in BERT and GPT-2 tokenizers as well, to allow for pre-caching of tokens	2019-03-05 12:31:45 -08:00
Catalin Voss	e99bc87e4d	Merge branch 'patch-1' into patch-2	2019-03-05 12:24:18 -08:00
Catalin Voss	9775b2eb27	Allow tokenization of sequences > 512 for caching ... For many applications requiring randomized data access, it's easier to cache the tokenized representations than the words. So why not turn this into a warning?	2019-03-02 16:30:21 -08:00
Catalin Voss	c0cf0a04d5	Fix typo	2019-02-27 18:01:06 -08:00
Thomas Wolf	2152bfeae8	Merge pull request #316 from joelgrus/gpt2docs ... update documentation for gpt-2	2019-02-24 09:38:29 +01:00
Joel Grus	8722e9eb3b	finish updating docstrings	2019-02-23 06:31:59 -08:00
Joel Grus	33aa7a80ca	update documentation	2019-02-22 15:37:59 -08:00
Thomas Wolf	a5b3a89545	Merge pull request #310 from spolu/spolu-nits_gpt2 ... Few small nits in GPT-2's README code examples	2019-02-21 10:23:27 +01:00
Stanislas Polu	ff22b3acc0	Few small nits in GPT-2's code examples	2019-02-21 09:15:27 +00:00
Thomas Wolf	cbb7fad319	Merge pull request #307 from guotong1988/patch-1 ... Update README.md	2019-02-21 09:25:19 +01:00
Tong Guo	09efcece75	Update README.md	2019-02-21 11:25:33 +08:00
Thomas Wolf	97c815dae2	Merge pull request #305 from bkj/patch-1 ... Update run_openai_gpt.py	2019-02-20 21:24:06 +01:00
Ben Johnson	8607233679	Update run_openai_gpt.py	2019-02-20 13:58:54 -05:00
Thomas Wolf	f50b82af04	Merge pull request #302 from yongbowin/master ... typo	2019-02-20 14:14:02 +01:00
Yongbo Wang	2fdab323d1	typo	2019-02-20 21:11:06 +08:00
Yongbo Wang	813e4d18ba	typo	2019-02-20 21:10:07 +08:00
Thomas Wolf	8337740754	Merge pull request #295 from tnlin/master ... fix broken link in readme	2019-02-19 14:00:28 +01:00
Tony Lin	5b0e0b61f0	fix typo in readme	2019-02-19 20:34:18 +08:00
Thomas Wolf	3ca35b99ba	Merge pull request #293 from davidefiocco/patch-2 ... Minor README typos corrected	2019-02-19 09:00:01 +01:00
Davide Fiocco	0ae8eece55	MInor README typos corrected	2019-02-18 21:28:28 +01:00
Thomas Wolf	07ebe0fd06	Merge pull request #292 from sam-qordoba/patch-3 ... Fix typo in `GPT2Model` code sample	2019-02-18 21:07:39 +01:00
sam-qordoba	1cb9c76ec5	Fix typo in GPT2Model code sample ... Typo prevented code from running	2019-02-18 09:27:26 -08:00
Thomas Wolf	a25d056b7a	update readme	2019-02-18 15:30:11 +01:00
Thomas Wolf	517d7c8624	update readme	2019-02-18 14:39:55 +01:00
Thomas Wolf	ada22a1c9e	more details in GPT-2 usage example	2019-02-18 14:37:41 +01:00
Thomas Wolf	522733f6cb	readme typo fixes	2019-02-18 14:32:10 +01:00
thomwolf	0202da0271	remove unnecessary example	2019-02-18 13:51:42 +01:00
Thomas Wolf	8f46cd1057	Merge pull request #288 from huggingface/gpt2 ... forgot to add regex to requirements.txt :(	2019-02-18 12:00:11 +01:00
thomwolf	e0855e8929	forgot to add regex to requirements :(	2019-02-18 11:54:51 +01:00
Thomas Wolf	0856a231c0	Merge pull request #287 from huggingface/gpt2 ... Gpt2	2019-02-18 11:38:05 +01:00
thomwolf	ab7f5d2943	simple	2019-02-18 11:33:54 +01:00
thomwolf	b450a7faf2	clean up tokenization - fix python 2 tests	2019-02-18 11:27:18 +01:00
thomwolf	d44db1145c	update readme	2019-02-18 11:12:09 +01:00
thomwolf	690a0dbf36	fix example - masking	2019-02-18 10:50:30 +01:00
thomwolf	fbb248a2e4	examples testing	2019-02-18 01:28:18 +01:00
thomwolf	5ff0c60505	language update	2019-02-18 00:55:47 +01:00
thomwolf	210d407245	updating init	2019-02-18 00:55:39 +01:00
thomwolf	b65f07d8c0	adding examples	2019-02-18 00:55:33 +01:00
thomwolf	009ee86a19	fix tests - bump up version	2019-02-17 23:57:23 +01:00
thomwolf	ffd623823d	adding gpt2	2019-02-17 23:38:51 +01:00
Thomas Wolf	3a2f97db6f	Merge pull request #286 from hendrycks/patch-1 ... Update activation function docstring	2019-02-17 15:30:46 +01:00
Dan Hendrycks	434d15da8e	Update activation function docstring	2019-02-16 12:17:52 -08:00
Thomas Wolf	5faf386652	Merge pull request #282 from wlhgtc/master ... Fix some bug about SQuAD code	2019-02-15 10:06:51 +01:00
wlhgtc	8efaf8f176	fix 'best_non_null_entry' is None error	2019-02-15 15:57:25 +08:00
Thomas Wolf	0e774e57a6	Update readme ... Adding details on how to extract a full list of hidden states for the Transformer-XL	2019-02-14 08:39:58 +01:00
Thomas Wolf	c35d9d48d9	Merge pull request #275 from davidefiocco/patch-1 ... --do_lower_case is duplicated in parser args	2019-02-13 16:32:21 +01:00
Davide Fiocco	65df0d78ed	--do_lower_case is duplicated in parser args ... Deleting one repetition (please review!)	2019-02-13 15:30:05 +01:00
Thomas Wolf	4e56da38d9	Merge pull request #268 from wangxiaodiu/master ... fixed a minor bug in README.md	2019-02-13 10:19:25 +01:00
Thomas Wolf	cdcb206e10	Merge pull request #273 from huggingface/update_to_fifth_release ... Update to fifth release	2019-02-13 10:19:08 +01:00
thomwolf	321d70a7a9	bump up to 0.5.1	2019-02-13 10:11:20 +01:00