Aymeric Augustin
d6eaf4e6d2
Update comments mentioning Python 2.
2019-12-22 18:38:56 +01:00
Aymeric Augustin
c824d15aa1
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
Aymeric Augustin
783a616999
Fix F401 flake8 warning (x88 / 116).
...
This change is mostly autogenerated with:
$ python -m autoflake --in-place --recursive --remove-all-unused-imports --ignore-init-module-imports examples templates transformers utils hubconf.py setup.py
I made minor changes in the generated diff.
2019-12-22 10:59:08 +01:00
Aymeric Augustin
631be27078
Fix E722 flake8 warnings (x26).
2019-12-22 10:59:07 +01:00
Aymeric Augustin
158e82e061
Sort imports with isort.
...
This is the result of:
$ isort --recursive examples templates transformers utils hubconf.py setup.py
2019-12-22 10:57:46 +01:00
Aymeric Augustin
fa84ae26d6
Reformat source code with black.
...
This is the result of:
$ black --line-length 119 examples templates transformers utils hubconf.py setup.py
There's a lot of fairly long lines in the project. As a consequence, I'm
picking the longest widely accepted line length, 119 characters.
This is also Thomas' preference, because it allows for explicit variable
names, to make the code easier to understand.
2019-12-21 17:52:29 +01:00
thomwolf
b03872aae0
fix merge
2019-12-21 14:49:54 +01:00
Thomas Wolf
518ba748e0
Merge branch 'master' into saving-and-resuming
2019-12-21 14:41:39 +01:00
Thomas Wolf
18601c3b6e
Merge pull request #2173 from erenup/master
...
run_squad with roberta
2019-12-21 14:33:16 +01:00
Thomas Wolf
eeb70cdd77
Merge branch 'master' into saving-and-resuming
2019-12-21 14:29:59 +01:00
thomwolf
8a2be93b4e
fix merge
2019-12-21 13:31:28 +01:00
Thomas Wolf
562f864038
Merge branch 'master' into fix-xlnet-squad2.0
2019-12-21 12:48:10 +01:00
Francesco
62c1fc3c1e
Removed duplicate XLMConfig, XLMForQuestionAnswering and XLMTokenizer from import statement of run_squad.py script
2019-12-19 09:50:56 -05:00
Lysandre
d803409215
Fix run squad evaluate during training
2019-12-16 16:31:38 -05:00
erenup
c7780700f5
Merge branch 'refs/heads/squad_roberta'
...
# Conflicts:
# transformers/data/processors/squad.py
2019-12-14 08:53:59 +08:00
erenup
8e9526b4b5
add multiple processing
2019-12-14 08:43:58 +08:00
Lysandre
c8ed1c82c8
[SQUAD] Load checkpoint when evaluating without training
2019-12-13 12:13:48 -05:00
erenup
9b312f9d41
initial version for roberta squad
2019-12-13 14:51:40 +08:00
LysandreJik
7296f1010b
Cleanup squad and add allow train_file and predict_file usage
2019-12-12 13:01:04 -05:00
Bilal Khan
fdc05cd68f
Update run_squad to save optimizer and scheduler states, then resume training from a checkpoint
2019-12-10 19:31:22 -06:00
LysandreJik
6a73382706
Complete warning + cleanup
2019-12-10 14:33:24 -05:00
Lysandre
dc4e9e5cb3
DataParallel for SQuAD + fix XLM
2019-12-10 19:21:20 +00:00
LysandreJik
2a4ef098d6
Add ALBERT and XLM to SQuAD script
2019-12-09 10:46:47 -05:00
Lysandre Debut
00c4e39581
Merge branch 'master' into squad-refactor
2019-12-09 10:41:15 -05:00
LysandreJik
e9217da5ff
Cleanup
...
Improve global visibility on the run_squad script, remove unused files and fixes related to XLNet.
2019-12-05 16:01:51 -05:00
LysandreJik
9ecd83dace
Patch evaluation for impossible values + cleanup
2019-12-05 14:44:57 -05:00
LysandreJik
f7e4a7cdfa
Cleanup
2019-12-04 16:24:15 -05:00
LysandreJik
cca75e7884
Kill the demon spawn
2019-12-04 15:42:29 -05:00
LysandreJik
9ddc3f1a12
Naming update + XLNet/XLM evaluation
2019-12-04 10:37:00 -05:00
LysandreJik
de276de1c1
Working evaluation
2019-12-03 17:15:51 -05:00
Ethan Perez
96e83506d1
Always use SequentialSampler during evaluation
...
When evaluating, shouldn't we always use the SequentialSampler instead of DistributedSampler? Evaluation only runs on 1 GPU no matter what, so if you use the DistributedSampler with N GPUs, I think you'll only evaluate on 1/N of the evaluation set. That's at least what I'm finding when I run an older/modified version of this repo.
2019-12-03 10:15:39 -05:00
Lysandre
bd41e8292a
Cleanup & Evaluation now works
2019-11-28 16:03:56 -05:00
Lysandre
4374eaea78
ALBERT for SQuAD
2019-11-26 13:08:12 -05:00
Lysandre
c3ba645237
Works for XLNet
2019-11-22 16:27:37 -05:00
Lysandre
72e506b22e
wip
2019-11-22 16:26:00 -05:00
Kazutoshi Shinoda
f3386d9383
typo "deay" -> "decay"
2019-11-18 11:50:06 -05:00
Xu Hongshen
7da3ef24cd
add is_impossible tensor to model inputs during fine-tuning xlnet on squad2.0
2019-11-15 14:18:53 +08:00
Thomas Wolf
9629e2c676
Merge pull request #1804 from ronakice/master
...
fix multi-gpu eval in torch examples
2019-11-14 22:24:05 +01:00
Rémi Louf
2276bf69b7
update the examples, docs and template
2019-11-14 20:38:02 +01:00
ronakice
2e31176557
fix multi-gpu eval
2019-11-12 05:55:11 -05:00
Thomas Wolf
ba973342e3
Merge pull request #1553 from WilliamTambellini/timeSquadInference
...
Add speed log to examples/run_squad.py
2019-11-05 11:13:12 +01:00
thomwolf
89d6272898
Fix #1623
2019-11-04 16:21:12 +01:00
Thomas Wolf
9a3b173cd3
Merge branch 'master' into master
2019-11-04 11:41:26 +01:00
Pasquale Minervini
3775550c4b
gradient norm clipping should be done right before calling the optimiser
2019-10-20 22:33:56 +01:00
William Tambellini
0919389d9a
Add speed log to examples/run_squad.py
...
Add a speed estimate log (time per example)
for evaluation to examples/run_squad.py
2019-10-17 14:41:04 -07:00
hlums
74c5035808
Fix token order in xlnet preprocessing.
2019-10-14 21:27:11 +00:00
Simon Layton
4e6a55751a
Force einsum to fp16
2019-10-14 11:12:41 -04:00
Bilal Khan
5ce8d29abe
Change tensorboard imports to use built-in tensorboard if available
2019-10-08 16:29:43 -05:00
VictorSanh
0820bb0555
unecessary carriage return
2019-10-04 17:23:15 -04:00
VictorSanh
f5891c3821
run_squad --> run_squad_w_distillation
2019-10-04 17:23:15 -04:00
