Commit Graph

15053 Commits

Author SHA1 Message Date
Julien Chaumond
9a50828b5c Pipelines: fix crash when modelcard is None
cc @mfuntowicz does this seem correct?
2020-02-03 17:53:39 -05:00
Lysandre
6c1b23554f Sample instead of greedy decoding by default in generate 2020-02-03 17:23:53 -05:00
Lysandre
239dd23f64 [Follow up 213]
Masked indices should have -1 and not -100. Updating documentation + scripts that were forgotten
2020-02-03 16:08:05 -05:00
Martin Malmsten
522c5b5533 Added README.md to Swedish BERT models from National Library of Sweden 2020-02-03 09:09:34 -05:00
Julien Plu
9329e59700 Add READMEs to Tensorflow versions of CamemBERT and XLM-RoBERTa 2020-02-03 09:04:34 -05:00
Antonio Carlos Falcão Petri
2ba147ecff Fix typo in examples/utils_ner.py
"%s-%d".format() -> "{}-{}".format()
2020-02-01 11:10:57 -05:00
Bram Vanroy
9773e5e0d9
CLI script to gather environment info (#2699)
* add "info" command to CLI

As a convenience, add the info directive to CLI. Running `python transformers-cli info` will return a string containing the transformers version, platform, python version, PT/TF version and GPU support

* Swap f-strings for .format

Still supporting 3.5 so can't use f-strings (sad face)

* Add reference in issue to CLI

* Add the expected fields to issue template

This way, people can still add the information manually if they want. (Though I fear they'll just ignore it.)

* Remove heading from output

* black-ify

* order of imports

Should ensure isort test passes

* use is_X_available over import..pass

* style

* fix copy-paste bug

* Rename command info -> env

Also adds the command to CONTRIBUTING.md in "Did you find a bug" section
2020-02-01 10:38:14 -05:00
Julien Chaumond
ddb6f9476b [model_cards] dbmdz models
Co-Authored-By: Stefan Schweter <stefan-it@users.noreply.github.com>
2020-01-31 18:39:09 -05:00
Julien Chaumond
6636826f04 [model_cards] Multilingual + Dutch SQuAD2.0
Co-Authored-By: HenrykBorzymowski <henrykborzymowski@users.noreply.github.com>
2020-01-31 18:39:09 -05:00
Julien Chaumond
98dadc98e1 [model_cards] UmBERTo
Co-Authored-By: Loreto Parisi <loretoparisi@gmail.com>
Co-Authored-By: Simone Francia <francia.simone1@gmail.com>
2020-01-31 18:39:09 -05:00
Julien Chaumond
d6fc34b459 [model_cards] add mine 2020-01-31 18:39:09 -05:00
Lysandre
d426b58b9e Patch: v2.4.1 2020-01-31 14:55:33 -05:00
Lysandre
1e82cd8457 Flaubert auto tokenizer + tests
cc @julien-c
2020-01-31 14:16:52 -05:00
Lysandre
d18d47be67 run_generation style 2020-01-31 12:05:48 -05:00
Lysandre
ff6f1492e8 FlauBERT load in AutoModel
The FlauBERT configuration file inherits from XLMConfig, and is recognized as such when loading from AutoModels as the XLMConfig is checked before the FlaubertConfig.

Changing the order solves this problem, but a test should be added.
2020-01-31 12:05:15 -05:00
Lysandre
7365f01d43 do_sample should be set to True in run_generation.py 2020-01-31 11:49:32 -05:00
Arnaud
3a21d6da6b Typo on markdown link in README.md 2020-01-31 10:58:49 -05:00
Lysandre
0aa40e9569 v2.4.0 documentation 2020-01-31 09:55:34 -05:00
Lysandre
8036ceb7c5 Update commands for pypi test 2020-01-31 09:48:15 -05:00
Lysandre
6664ea943d Release: v2.4.0 2020-01-31 09:40:32 -05:00
Julien Chaumond
5a6b138b00
[Umberto] model shortcuts (#2661)
* [Umberto] model shortcuts

cc @loretoparisi @simonefrancia

see #2485

* Ensure that tokenizers will be correctly configured
2020-01-30 21:05:53 -05:00
Julien Chaumond
7fe294bf07 Hotfix: same handling of non-existent files as for config 2020-01-30 20:05:04 -05:00
Julien Chaumond
b85c59f997 config.architectures 2020-01-30 19:26:59 -05:00
Julien Chaumond
f9bc3f5771 style tweak 2020-01-30 19:26:59 -05:00
Julien Chaumond
0b13fb822a No need for a model_type here
cc @lysandrejik
2020-01-30 19:26:59 -05:00
Jared Nielsen
71a382319f Correct documentation 2020-01-30 18:41:24 -05:00
Lysandre
01a14ebd8d Add FlauBERT to automodels 2020-01-30 18:40:22 -05:00
Julien Chaumond
9fa836a73f
fill_mask helper (#2576)
* fill_mask helper

* [poc] FillMaskPipeline

* Revert "[poc] FillMaskPipeline"

This reverts commit 67eeea55b0.

* Revert "fill_mask helper"

This reverts commit cacc17b884.

* README: clarify that Pipelines can also do text-classification

cf. question at the AI&ML meetup last week, @mfuntowicz

* Fix test: test feature-extraction pipeline

* Test tweaks

* Slight refactor of existing pipeline (in preparation of new FillMaskPipeline)

* Extraneous doc

* More robust way of doing this

@mfuntowicz as we don't rely on the model name anymore (see AutoConfig)

* Also add RobertaConfig as a quickfix for wrong token_type_ids

* cs

* [BIG] FillMaskPipeline
2020-01-30 18:15:42 -05:00
Hang Le
b43cb09aaa Add layerdrop 2020-01-30 12:05:01 -05:00
Lysandre
df27648bd9 Rename test_examples to test_doc_samples 2020-01-30 10:07:22 -05:00
Lysandre
93dccf527b Pretrained models 2020-01-30 10:04:18 -05:00
Lysandre
90787fed81 Style 2020-01-30 10:04:18 -05:00
Lysandre
73306d028b FlauBERT documentation 2020-01-30 10:04:18 -05:00
Lysandre
ce2f4227ab Fix failing FlauBERT test 2020-01-30 10:04:18 -05:00
Hang Le
f0a4fc6cd6 Add Flaubert 2020-01-30 10:04:18 -05:00
Peter Izsak
a5381495e6 Added classifier dropout rate in ALBERT 2020-01-30 09:52:34 -05:00
Bram Vanroy
83446a88d9 Use _pad_token of pad_token_id
Requesting pad_token_id would cause an error message when it is None. Use private _pad_token instead.
2020-01-29 17:44:58 -05:00
BramVanroy
9fde13a3ac Add check to verify existence of pad_token_id
In batch_encode_plus we have to ensure that the tokenizer has a pad_token_id so that, when padding, no None values are added as padding. That would happen with gpt2, openai, transfoxl.

closes https://github.com/huggingface/transformers/issues/2640
2020-01-29 17:44:58 -05:00
Lysandre
e63a81dd25 Style 2020-01-29 16:29:20 -05:00
Lysandre
217349016a Copy object instead of passing the reference 2020-01-29 16:15:39 -05:00
Jared Nielsen
adb8c93134 Remove lines causing a KeyError 2020-01-29 14:01:16 -05:00
Lysandre
c69b082601 Update documentation 2020-01-29 12:06:13 -05:00
Julien Plu
ca1d66734d Apply quality and style requirements once again 2020-01-29 12:06:13 -05:00
Julien Plu
5e3c72842d bugfix on model name 2020-01-29 12:06:13 -05:00
Julien Plu
0731fa1587 Apply quality and style requirements 2020-01-29 12:06:13 -05:00
Julien Plu
a3998e76ae Add TF2 CamemBERT model 2020-01-29 12:06:13 -05:00
Lysandre
b5625f131d Style 2020-01-29 11:47:49 -05:00
Lysandre
44a5b4bbe7 Update documentation 2020-01-29 11:47:49 -05:00
Julien Plu
7fc628d98e Apply style 2020-01-29 11:47:49 -05:00
Julien Plu
64ca855617 Add TF2 XLM-RoBERTa model 2020-01-29 11:47:49 -05:00