Aymeric Augustin
d6eaf4e6d2
Update comments mentioning Python 2.
2019-12-22 18:38:56 +01:00
Aymeric Augustin
c824d15aa1
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
Aymeric Augustin
fa2ccbc081
Fix E266 flake8 warning (x90).
2019-12-22 10:59:08 +01:00
Aymeric Augustin
631be27078
Fix E722 flake8 warnings (x26).
2019-12-22 10:59:07 +01:00
Aymeric Augustin
158e82e061
Sort imports with isort.
...
This is the result of:
$ isort --recursive examples templates transformers utils hubconf.py setup.py
2019-12-22 10:57:46 +01:00
Aymeric Augustin
fa84ae26d6
Reformat source code with black.
...
This is the result of:
$ black --line-length 119 examples templates transformers utils hubconf.py setup.py
There's a lot of fairly long lines in the project. As a consequence, I'm
picking the longest widely accepted line length, 119 characters.
This is also Thomas' preference, because it allows for explicit variable
names, to make the code easier to understand.
2019-12-21 17:52:29 +01:00
Thomas Wolf
6f68d559ab
Merge pull request #2130 from huggingface/ignored-index-coherence
...
[BREAKING CHANGE] Setting all ignored index to the PyTorch standard
2019-12-21 14:55:40 +01:00
Julien Chaumond
a5a06a851e
[doc] Param name consistency
2019-12-19 16:24:20 -05:00
Aidan Kierans
1718fb9e74
Minor/basic text fixes ( #2229 )
...
* Small clarification
Matches line 431 to line 435 for additional clarity and consistency.
* Fixed minor typo
The letter "s" was previously omitted from the word "docstrings".
2019-12-19 16:23:18 -05:00
LysandreJik
b72f9d340e
Correct index in script
2019-12-10 18:33:17 -05:00
Bilal Khan
79526f82f5
Remove unnecessary epoch variable
2019-12-09 16:24:35 -05:00
Bilal Khan
9626e0458c
Add functionality to continue training from last saved global_step
2019-12-09 16:24:35 -05:00
Bilal Khan
2d73591a18
Stop saving current epoch
2019-12-09 16:24:35 -05:00
Bilal Khan
0eb973b0d9
Use saved optimizer and scheduler states if available
2019-12-09 16:24:35 -05:00
Bilal Khan
a03fcf570d
Save tokenizer after each epoch to be able to resume training from a checkpoint
2019-12-09 16:24:35 -05:00
Bilal Khan
f71b1bb05a
Save optimizer state, scheduler state and current epoch
2019-12-09 16:24:35 -05:00
thomwolf
5bfcd0485e
fix #1991
2019-12-04 14:53:11 +01:00
VictorSanh
48cbf267c9
Use full dataset for eval (SequentialSampler in Distributed setting)
2019-12-03 11:01:37 -05:00
maxvidal
b0ee7c7df3
Added Camembert to available models
2019-11-29 14:17:02 -05:00
İbrahim Ethem Demirci
aa92a184d2
resize model when special tokenizer present
2019-11-25 15:06:32 -05:00
Lysandre
7485caefb0
fix #1894
2019-11-25 09:33:39 -05:00
Thomas Wolf
9629e2c676
Merge pull request #1804 from ronakice/master
...
fix multi-gpu eval in torch examples
2019-11-14 22:24:05 +01:00
Thomas Wolf
df99f8c5a1
Merge pull request #1832 from huggingface/memory-leak-schedulers
...
replace LambdaLR scheduler wrappers by function
2019-11-14 22:10:31 +01:00
Rémi Louf
2276bf69b7
update the examples, docs and template
2019-11-14 20:38:02 +01:00
Lysandre
d7929899da
Specify checkpoint in saved file for run_lm_finetuning.py
2019-11-14 10:49:00 -05:00
ronakice
2e31176557
fix multi-gpu eval
2019-11-12 05:55:11 -05:00
thomwolf
89d6272898
Fix #1623
2019-11-04 16:21:12 +01:00
altsoph
079bfb32fb
Evaluation fixed.
2019-10-28 10:18:58 -04:00
altsoph
438f2730a0
Evaluation code fixed.
2019-10-28 10:18:58 -04:00
Luran He
f382a8decd
convert int to str before adding to a str
2019-10-10 19:20:39 -04:00
Thomas Wolf
6596e3d566
Merge pull request #1454 from bkkaggle/pytorch-built-in-tensorboard
...
Change tensorboard imports to use built-in tensorboard if available
2019-10-10 11:56:55 +02:00
Lysandre Debut
e84470ef81
Merge pull request #1384 from huggingface/encoding-qol
...
Quality of life enhancements in encoding + patch MLM masking
2019-10-09 11:18:24 -04:00
jinoobaek-qz
69629c4f0f
Improve naming and only do regex when necessary
2019-10-09 08:48:40 -04:00
jinoobaek-qz
bf34a252b8
Golden path
2019-10-09 08:48:40 -04:00
jinoobaek-qz
528d3f327b
Improve readability and improve make less assumptions about checkpoint format
2019-10-09 08:48:40 -04:00
jinoobaek-qz
56301bd9e8
Extract method
2019-10-09 08:48:40 -04:00
jinoobaek-qz
d6c5469712
Delete older checkpoint after saving new checkpoint
2019-10-09 08:48:40 -04:00
jinoobaek-qz
54a31f50fb
Add save_total_limit
2019-10-09 08:48:40 -04:00
Bilal Khan
5ce8d29abe
Change tensorboard imports to use built-in tensorboard if available
2019-10-08 16:29:43 -05:00
thomwolf
6c1d0bc066
update encode_plus - add truncation strategies
2019-10-04 17:38:38 -04:00
LysandreJik
aebd83230f
Update naming + remove f string in run_lm_finetuning example
2019-10-03 11:31:36 -04:00
LysandreJik
5ed50a93fb
LM finetuning won't mask special tokens anymore
2019-10-03 11:31:36 -04:00
Brian Ma
2195c0d5f9
Evaluation result.txt path changing #1286
2019-10-03 12:49:12 +08:00
Thomas Wolf
963529e29b
Merge pull request #1288 from echan00/master
...
Typo with LM Fine tuning script
2019-10-01 18:46:07 -04:00
thomwolf
f7978f70ec
use format instead of f-strings
2019-10-01 18:45:38 -04:00
Denny
9478590630
Update run_lm_finetuning.py
...
The previous method, just as phrased, did not exist in the class.
2019-09-27 15:18:42 -03:00
mgrankin
f71a4577b8
faster dataset building
2019-09-26 16:53:13 +03:00
thomwolf
31c23bd5ee
[BIG] pytorch-transformers => transformers
2019-09-26 10:15:53 +02:00
LysandreJik
bf503158c5
Sentence -> Sequence. Removed output_mask from the special token addition methods.
2019-09-19 10:55:06 +02:00
LysandreJik
88368c2a16
Added DistilBERT to run_lm_finetuning
2019-09-19 10:55:06 +02:00
