karajan1001
|
f01b3e6680
|
fix #2399 an ImportError in official example (#2400)
* fix #2399 an ImportError in official example
* style
Co-authored-by: Julien Chaumond <chaumond@gmail.com>
|
2020-01-05 12:50:20 -05:00 |
|
Julien Chaumond
|
78528742f1
|
Fix syntax + link to community page
|
2020-01-05 12:43:39 -05:00 |
|
Clement
|
12e0aa4368
|
Proposition to include community models in readme
|
2020-01-05 12:37:11 -05:00 |
|
Morgan Funtowicz
|
80faf22b4a
|
Updating documentation for converting tensorflow model to reflect the new cli convert format.
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
|
2020-01-04 13:41:18 +01:00 |
|
Julien Chaumond
|
629b22adcf
|
[run_lm_finetuning] mask_tokens: document types
|
2020-01-01 12:55:10 -05:00 |
|
Julien Chaumond
|
594ca6dead
|
[debug] Debug Heisenbug, the old school way.
|
2019-12-29 10:07:21 -05:00 |
|
Julien Chaumond
|
0df4e62da0
|
[http] Tweak http user-agent (#2353)
|
2019-12-29 10:06:50 -05:00 |
|
Thomas Wolf
|
f75bf05ce6
|
Merge pull request #2352 from huggingface/cli_tweaks
Cli tweaks
|
2019-12-28 15:40:00 +01:00 |
|
Julien Chaumond
|
0d467fd6de
|
Typo
|
2019-12-27 23:06:48 -05:00 |
|
Julien Chaumond
|
d8293e84f3
|
[cli] upload: max number of files at the same time
|
2019-12-27 23:02:53 -05:00 |
|
Julien Chaumond
|
4d6c93e923
|
Kill __main__
|
2019-12-27 22:55:22 -05:00 |
|
Julien Chaumond
|
9b2badf3c9
|
[cli] Update doc
|
2019-12-27 22:54:29 -05:00 |
|
Julien Chaumond
|
f78ebc22ad
|
[cli] Add ability to delete remote object
|
2019-12-27 22:53:49 -05:00 |
|
Anthony MOI
|
bfe870be65
|
Hotfix tokenizers version for sdist installs
|
2019-12-27 11:05:52 -05:00 |
|
Thomas Wolf
|
74ea432847
|
Merge pull request #2286 from adelevie/patch-2
Typo in tokenization_utils.py
|
2019-12-27 10:50:47 +01:00 |
|
Thomas Wolf
|
492bea9aa0
|
Merge pull request #2292 from patrickvonplaten/add_cached_past_for_language_generation
Add cached past for language generation
|
2019-12-27 10:33:27 +01:00 |
|
Thomas Wolf
|
e213900fa2
|
Merge pull request #2290 from patrickvonplaten/fix_typo_in_doc_for_language_generation
duplicated line for repeating_words_penalty_for_language_generation
|
2019-12-27 10:29:06 +01:00 |
|
Thomas Wolf
|
9f5f646442
|
Merge pull request #2211 from huggingface/fast-tokenizers
Fast tokenizers
|
2019-12-27 10:24:29 +01:00 |
|
Aymeric Augustin
|
9024b19994
|
Auto-format (fixes previous commit).
|
2019-12-27 10:13:52 +01:00 |
|
Aymeric Augustin
|
3233b58ad4
|
Quote square brackets in shell commands.
This ensures compatibility with zsh.
Fix #2316.
|
2019-12-27 08:50:25 +01:00 |
|
Anthony MOI
|
e6ec24fa88
|
Better added_tokens handling
|
2019-12-26 16:49:48 -05:00 |
|
Anthony MOI
|
599db139f9
|
Code style update
|
2019-12-26 15:13:30 -05:00 |
|
Anthony MOI
|
835b76a46f
|
Handle unk_token
As we discussed, this is handled here directly
cc @thomwolf
|
2019-12-26 14:42:55 -05:00 |
|
Anthony MOI
|
7ead04ce14
|
FastPreTrainedTokenizer => PreTrainedTokenizerFast
|
2019-12-26 14:39:39 -05:00 |
|
Anthony MOI
|
1f82a5d910
|
Update for changes in tokenizers API
|
2019-12-26 14:37:55 -05:00 |
|
Thomas Wolf
|
8c67b529f6
|
Merge pull request #2324 from kashif/patch-1
Typo in serving.py
|
2019-12-26 12:38:06 +01:00 |
|
Kashif Rasul
|
7211541ade
|
Typo in serving.py
|
2019-12-26 12:21:40 +01:00 |
|
patrickvonplaten
|
0f6017bee3
|
improve comments for examples
|
2019-12-26 00:35:11 +01:00 |
|
patrickvonplaten
|
87c8fca9bc
|
add example for ctrl text generation in docs
|
2019-12-26 00:29:19 +01:00 |
|
patrickvonplaten
|
88def24c45
|
merge conflicts - renamed to previous_token singular
|
2019-12-26 00:27:16 +01:00 |
|
patrickvonplaten
|
822f725a07
|
duplicated line for repeating_words_penalty_for_language_generation
|
2019-12-26 00:25:29 +01:00 |
|
patrickvonplaten
|
fc84bd5254
|
adapt style to predefined style layout
|
2019-12-25 23:32:44 +01:00 |
|
patrickvonplaten
|
deff792bb6
|
add prepare inputs for transfo_xl and xlnet
|
2019-12-25 23:17:24 +01:00 |
|
patrickvonplaten
|
9398058e19
|
add easy tensor shape match test
|
2019-12-25 23:17:24 +01:00 |
|
patrickvonplaten
|
90cda45e9e
|
add past re-ordering for beam search
|
2019-12-25 23:17:24 +01:00 |
|
patrickvonplaten
|
6bca56fdb0
|
check for self.config.mem_len instead of self.mem_len in _do_output_past
|
2019-12-25 23:17:24 +01:00 |
|
patrickvonplaten
|
365ccd0af2
|
make if statements cleaner for prepare_inputs_for_generation
|
2019-12-25 23:17:24 +01:00 |
|
patrickvonplaten
|
d039c679d2
|
better naming for if statement
|
2019-12-25 23:17:24 +01:00 |
|
patrickvonplaten
|
7e0c5c731a
|
changed do_output_past function to check for self.config.output_past instead of self.output_past
|
2019-12-25 23:17:24 +01:00 |
|
patrickvonplaten
|
eeaa402cd4
|
rename comments
|
2019-12-25 23:17:24 +01:00 |
|
patrickvonplaten
|
7bb4271291
|
remove ipdb debugging statements
|
2019-12-25 23:17:24 +01:00 |
|
patrickvonplaten
|
267587c258
|
add and improve comments
|
2019-12-25 23:17:24 +01:00 |
|
patrickvonplaten
|
d891fd0ae0
|
add past hidden key states for more efficient language generation & add prepare_inputs for gpt2 and ctrl model
|
2019-12-25 23:17:24 +01:00 |
|
Thomas Wolf
|
aeef4823ab
|
Merge pull request #2303 from patrickvonplaten/fix_error_with_repetition_penalty
fix repetition penalty error in modeling_utils.py
|
2019-12-25 22:39:20 +01:00 |
|
Thomas Wolf
|
0412f3d929
|
Merge pull request #2291 from aaugustin/fix-flake8-F841
Fix F841 flake8 warning
|
2019-12-25 22:37:42 +01:00 |
|
Thomas Wolf
|
8742c95461
|
Merge pull request #2289 from patrickvonplaten/fix_effective_batch_size_lang_gen_xlm
fix bug in prepare inputs for language generation for xlm for effective batch_size > 1
|
2019-12-25 22:30:46 +01:00 |
|
Thomas Wolf
|
1240be3ed9
|
Merge pull request #2312 from vitaliyradchenko/fix_special_and_add_tokens_loading
Correct tokenization for special and added tokens
|
2019-12-25 20:52:30 +01:00 |
|
vitaliyradchenko
|
b262577d17
|
add special tokens to unique_added_tokens_encoder
|
2019-12-25 18:31:35 +02:00 |
|
vitaliyradchenko
|
83a2347952
|
fixed lack of added and special tokens
|
2019-12-25 18:03:19 +02:00 |
|
Thomas Wolf
|
cea04a2443
|
Merge pull request #2310 from ShnitzelKiller/scatter-unfix
revert erroneous fix #2276
|
2019-12-25 12:43:22 +01:00 |
|