thomwolf
cfa0380515
Merge branch 'master' into generation_sampler
2019-12-21 14:12:52 +01:00
thomwolf
300ec3003c
fixing run_generation example - using torch.no_grad
2019-12-21 14:02:19 +01:00
thomwolf
1c37746892
fixing run_generation
2019-12-21 13:52:49 +01:00
Thomas Wolf
59941c5d1f
Merge pull request #2189 from stefan-it/xlmr
...
Add support for XLM-RoBERTa
2019-12-20 13:26:38 +01:00
Julien Chaumond
a5a06a851e
[doc] Param name consistency
2019-12-19 16:24:20 -05:00
Aidan Kierans
1718fb9e74
Minor/basic text fixes ( #2229 )
...
* Small clarification
Matches line 431 to line 435 for additional clarity and consistency.
* Fixed minor typo
The letter "s" was previously omitted from the word "docstrings".
2019-12-19 16:23:18 -05:00
Francesco
62c1fc3c1e
Removed duplicate XLMConfig, XLMForQuestionAnswering and XLMTokenizer from import statement of run_squad.py script
2019-12-19 09:50:56 -05:00
Ejar
284572efc0
Updated typo on the link
...
Updated documentation due to typo
2019-12-19 09:36:43 -05:00
Stefan Schweter
a26ce4dee1
examples: add XLM-RoBERTa to glue script
2019-12-19 02:23:01 +01:00
thomwolf
3d2096f516
further cleanup
2019-12-18 11:50:54 +01:00
thomwolf
83bc5235cf
Merge branch 'master' into pr/2189
2019-12-17 11:47:32 +01:00
Thomas Wolf
f061606277
Merge pull request #2164 from huggingface/cleanup-configs
...
[SMALL BREAKING CHANGE] Cleaning up configuration classes - Adding Model Cards
2019-12-17 09:10:16 +01:00
Lysandre
18a879f475
fix #2180
2019-12-16 16:44:29 -05:00
Lysandre
d803409215
Fix run squad evaluate during training
2019-12-16 16:31:38 -05:00
Stefan Schweter
71b4750517
examples: add support for XLM-RoBERTa to run_ner script
2019-12-16 16:37:27 +01:00
thomwolf
7140363e09
update bertabs
2019-12-14 09:44:53 +01:00
Thomas Wolf
a52d56c8d9
Merge branch 'master' into cleanup-configs
2019-12-14 09:43:07 +01:00
Lysandre
c8ed1c82c8
[SQUAD] Load checkpoint when evaluating without training
2019-12-13 12:13:48 -05:00
Pierric Cistac
5a5c4349e8
Fix summarization to_cpu doc
2019-12-13 10:02:33 -05:00
thomwolf
47f0e3cfb7
cleaning up configuration classes
2019-12-13 14:33:24 +01:00
LysandreJik
7296f1010b
Cleanup squad and add allow train_file and predict_file usage
2019-12-12 13:01:04 -05:00
Alan deLevie
fbf5455a86
Fix typo in examples/run_glue.py args declaration.
...
deay -> decay
2019-12-12 11:16:19 -05:00
LysandreJik
6a73382706
Complete warning + cleanup
2019-12-10 14:33:24 -05:00
Lysandre
dc4e9e5cb3
DataParallel for SQuAD + fix XLM
2019-12-10 19:21:20 +00:00
Rémi Louf
07bc8efbc3
add greedy decoding and sampling
2019-12-10 17:27:50 +01:00
Rémi Louf
4b82c485de
remove misplaced summarization documentation
2019-12-10 09:13:33 -05:00
Thomas Wolf
e57d00ee10
Merge pull request #1984 from huggingface/squad-refactor
...
[WIP] Squad refactor
2019-12-10 11:07:26 +01:00
Julien Chaumond
1d18930462
Harmonize no_cuda flag with other scripts
2019-12-09 20:37:55 -05:00
Rémi Louf
f7eba09007
clean for release
2019-12-09 20:37:55 -05:00
Rémi Louf
2a64107e44
improve device usage
2019-12-09 20:37:55 -05:00
Rémi Louf
c0707a85d2
add README
2019-12-09 20:37:55 -05:00
Rémi Louf
ade3cdf5ad
integrate ROUGE
2019-12-09 20:37:55 -05:00
Rémi Louf
076602bdc4
prevent BERT weights from being downloaded twice
2019-12-09 20:37:55 -05:00
Rémi Louf
a1994a71ee
simplified model and configuration
2019-12-09 20:37:55 -05:00
Rémi Louf
3a9a9f7861
default output dir to documents dir
2019-12-09 20:37:55 -05:00
Rémi Louf
693606a75c
update the docs
2019-12-09 20:37:55 -05:00
Rémi Louf
2403a66598
give transformers API to BertAbs
2019-12-09 20:37:55 -05:00
Rémi Louf
ba089c780b
share pretrained embeddings
2019-12-09 20:37:55 -05:00
Rémi Louf
9660ba1cbd
Add beam search
2019-12-09 20:37:55 -05:00
Rémi Louf
1c71ecc880
load the pretrained weights for encoder-decoder
...
We currently save the pretrained_weights of the encoder and decoder in
two separate directories `encoder` and `decoder`. However, for the
`from_pretrained` function to operate with automodels we need to
specify the type of model in the path to the weights.
The path to the encoder/decoder weights is handled by the
`PreTrainedEncoderDecoder` class in the `save_pretrained` function. Sice
there is no easy way to infer the type of model that was initialized for
the encoder and decoder we add a parameter `model_type` to the function.
This is not an ideal solution as it is error prone, and the model type
should be carried by the Model classes somehow.
This is a temporary fix that should be changed before merging.
2019-12-09 20:37:55 -05:00
Rémi Louf
07f4cd73f6
update function to add special tokens
...
Since I started my PR the `add_special_token_single_sequence` function
has been deprecated for another; I replaced it with the new function.
2019-12-09 20:37:55 -05:00
Bilal Khan
79526f82f5
Remove unnecessary epoch variable
2019-12-09 16:24:35 -05:00
Bilal Khan
9626e0458c
Add functionality to continue training from last saved global_step
2019-12-09 16:24:35 -05:00
Bilal Khan
2d73591a18
Stop saving current epoch
2019-12-09 16:24:35 -05:00
Bilal Khan
0eb973b0d9
Use saved optimizer and scheduler states if available
2019-12-09 16:24:35 -05:00
Bilal Khan
a03fcf570d
Save tokenizer after each epoch to be able to resume training from a checkpoint
2019-12-09 16:24:35 -05:00
Bilal Khan
f71b1bb05a
Save optimizer state, scheduler state and current epoch
2019-12-09 16:24:35 -05:00
LysandreJik
2a4ef098d6
Add ALBERT and XLM to SQuAD script
2019-12-09 10:46:47 -05:00
Lysandre Debut
00c4e39581
Merge branch 'master' into squad-refactor
2019-12-09 10:41:15 -05:00
Thomas Wolf
5482822a2b
Merge pull request #2046 from jplu/tf2-ner-example
...
Add NER TF2 example.
2019-12-06 12:12:22 +01:00
