31 Commits

Author	SHA1	Message	Date
Thomas Wolf	3b56427a1e	Merge pull request #1040 from FeiWang96/multi_gpu ... Fix bug of multi-gpu training in lm finetuning	2019-08-20 17:13:44 +02:00
Duzeyao	d86b49ac86	swap optimizer.step and scheduler.step	2019-08-20 16:46:34 +08:00
Duzeyao	45ab8bf60e	Revert "Update finetune_on_pregenerated.py" ... This reverts commit a1359b970c.	2019-08-20 16:40:39 +08:00
Zeyao Du	a1359b970c	Update finetune_on_pregenerated.py	2019-08-20 16:00:07 +08:00
wangfei	856a63da4d	Fix: save model/model.module	2019-08-18 11:03:47 +08:00
wangfei	1ef41b8337	Revert "Fix: save model/model.module" ... This reverts commit 00e9c4cc96.	2019-08-18 11:03:12 +08:00
wangfei	00e9c4cc96	Fix: save model/model.module	2019-08-18 11:02:02 +08:00
wangfei	b8ff56896c	Fix bug of multi-gpu training in lm finetuning	2019-08-16 12:11:05 +08:00
yzy5630	a1fe4ba9c9	use new API for save and load	2019-07-18 15:45:23 +08:00
yzy5630	a7ba27b1b4	add parser for adam	2019-07-18 08:52:51 +08:00
yzy5630	d6522e2873	change loss and optimizer to new API	2019-07-17 21:22:34 +08:00
yzy5630	123da5a2fa	fix errors for lm_finetuning examples	2019-07-17 09:56:07 +08:00
thomwolf	2397f958f9	updating examples and doc	2019-07-14 23:20:10 +02:00
thomwolf	0bab55d5d5	[BIG] name change	2019-07-05 11:55:36 +02:00
thomwolf	c41f2bad69	WIP XLM + refactoring	2019-07-03 22:54:39 +02:00
Oliver Guhr	5c08c8c273	adds the tokenizer + model config to the output	2019-06-11 13:46:33 +02:00
burcturkoglu	00c7fd2b79	Division to num_train_optimizer of global_step in lr_this_step is removed.	2019-05-09 10:57:03 +03:00
burcturkoglu	5289b4b9e0	Division to num_train_optimizer of global_step in lr_this_step is removed.	2019-05-09 10:51:38 +03:00
thomwolf	d94c6b0144	fix training schedules in examples to match new API	2019-04-23 11:17:06 +02:00
Matthew Carrigan	b8e2a9c584	Made --reduce_memory actually do something in finetune_on_pregenerated	2019-04-22 14:01:48 +01:00
thomwolf	4bc4c69af9	finetuning any BERT model - fixes #455	2019-04-11 16:57:59 +02:00
Matthew Carrigan	8d1d1ffde2	Corrected the displayed loss when gradient_accumulation_steps > 1	2019-03-25 12:15:19 +00:00
Matthew Carrigan	7d1ae644ef	Added a --reduce_memory option to the training script to keep training ... data on disc as a memmap rather than in memory	2019-03-21 17:02:18 +00:00
Matthew Carrigan	8a861048dd	Fixed up the notes on a possible future low-memory path	2019-03-21 14:08:39 +00:00
Matthew Carrigan	a8a577ba93	Reduced memory usage for pregenerating the data a lot by writing it ... out on the fly without shuffling - the Sampler in the finetuning script will shuffle for us.	2019-03-21 14:05:52 +00:00
Matthew Carrigan	0ae59e662d	Reduced memory usage for pregenerating the data a lot by writing it ... out on the fly without shuffling - the Sampler in the finetuning script will shuffle for us.	2019-03-21 14:04:17 +00:00
Matthew Carrigan	7de5c6aa5e	PEP8 and formatting cleanups	2019-03-20 16:44:04 +00:00
Matthew Carrigan	1798e98e5a	Added final TODOs	2019-03-20 16:42:37 +00:00
Matthew Carrigan	c64c2fc4c2	Fixed embarrassing indentation problem	2019-03-20 15:42:57 +00:00
Matthew Carrigan	0540d360f2	Fixed logging	2019-03-20 15:36:51 +00:00
Matthew Carrigan	976554a472	First commit of the new LM finetuning	2019-03-20 14:23:51 +00:00