11 Commits

Author	SHA1	Message	Date
Joel Grus	ae152cec09	make save_pretrained work with added tokens ... right now it's dumping the *decoder* when it should be dumping the *encoder*. this fixes that.	2019-07-24 16:54:48 -07:00
thomwolf	1849aa7d39	update readme and pretrained model weight files	2019-07-16 15:11:29 +02:00
thomwolf	1b35d05d4b	update conversion scripts and __main__	2019-07-16 09:41:55 +02:00
thomwolf	15d8b1266c	update tokenizer - update squad example for xlnet	2019-07-15 17:30:42 +02:00
thomwolf	7d4b200e40	good quality generation example for GPT, GPT-2, Transfo-XL, XLNet	2019-07-13 15:25:03 +02:00
thomwolf	d5481cbe1b	adding tests to examples - updating summary module - coverage update	2019-07-09 15:29:42 +02:00
thomwolf	c079d7ddff	fix python 2 tests	2019-07-09 10:40:59 +02:00
thomwolf	b19786985d	unified tokenizer api and serialization + tests	2019-07-09 10:25:18 +02:00
thomwolf	1113f97f33	clean up glue example	2019-07-05 16:31:13 +02:00
thomwolf	6dacc79d39	fix python2 tests	2019-07-05 15:11:59 +02:00
thomwolf	36bca545ff	tokenization abstract class - tests for examples	2019-07-05 15:02:59 +02:00