riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-31 02:02:21 +06:00
Code Issues Actions 33 Packages Projects Releases Wiki Activity
3,141 Commits 887 Branches 232 Tags 2.1 GiB
1e82cd8457
Commit Graph

3141 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Lysandre
1e82cd8457 Flaubert auto tokenizer + tests 
cc @julien-c
 2020-01-31 14:16:52 -05:00
Lysandre
d18d47be67 run_generation style 2020-01-31 12:05:48 -05:00
Lysandre
ff6f1492e8 FlauBERT load in AutoModel 
The FlauBERT configuration file inherits from XLMConfig, and is recognized as such when loading from AutoModels as the XLMConfig is checked before the FlaubertConfig.

Changing the order solves this problem, but a test should be added.
 2020-01-31 12:05:15 -05:00
Lysandre
7365f01d43 do_sample should be set to True in run_generation.py 2020-01-31 11:49:32 -05:00
Arnaud
3a21d6da6b Typo on markdown link in README.md 2020-01-31 10:58:49 -05:00
Lysandre
0aa40e9569 v2.4.0 documentation 2020-01-31 09:55:34 -05:00
Lysandre
8036ceb7c5 Update commands for pypi test 2020-01-31 09:48:15 -05:00
Lysandre
6664ea943d Release: v2.4.0 2020-01-31 09:40:32 -05:00
Julien Chaumond
5a6b138b00
[Umberto] model shortcuts (#2661) 
* [Umberto] model shortcuts

cc @loretoparisi @simonefrancia

see #2485

* Ensure that tokenizers will be correctly configured
 2020-01-30 21:05:53 -05:00
Julien Chaumond
7fe294bf07 Hotfix: same handling of non-existent files as for config 2020-01-30 20:05:04 -05:00
Julien Chaumond
b85c59f997 config.architectures 2020-01-30 19:26:59 -05:00
Julien Chaumond
f9bc3f5771 style tweak 2020-01-30 19:26:59 -05:00
Julien Chaumond
0b13fb822a No need for a model_type here 
cc @lysandrejik
 2020-01-30 19:26:59 -05:00
Jared Nielsen
71a382319f Correct documentation 2020-01-30 18:41:24 -05:00
Lysandre
01a14ebd8d Add FlauBERT to automodels 2020-01-30 18:40:22 -05:00
Julien Chaumond
9fa836a73f
fill_mask helper (#2576) 
* fill_mask helper

* [poc] FillMaskPipeline

* Revert "[poc] FillMaskPipeline"

This reverts commit 67eeea55b0.

* Revert "fill_mask helper"

This reverts commit cacc17b884.

* README: clarify that Pipelines can also do text-classification

cf. question at the AI&ML meetup last week, @mfuntowicz

* Fix test: test feature-extraction pipeline

* Test tweaks

* Slight refactor of existing pipeline (in preparation of new FillMaskPipeline)

* Extraneous doc

* More robust way of doing this

@mfuntowicz as we don't rely on the model name anymore (see AutoConfig)

* Also add RobertaConfig as a quickfix for wrong token_type_ids

* cs

* [BIG] FillMaskPipeline
 2020-01-30 18:15:42 -05:00
Hang Le
b43cb09aaa Add layerdrop 2020-01-30 12:05:01 -05:00
Lysandre
df27648bd9 Rename test_examples to test_doc_samples 2020-01-30 10:07:22 -05:00
Lysandre
93dccf527b Pretrained models 2020-01-30 10:04:18 -05:00
Lysandre
90787fed81 Style 2020-01-30 10:04:18 -05:00
Lysandre
73306d028b FlauBERT documentation 2020-01-30 10:04:18 -05:00
Lysandre
ce2f4227ab Fix failing FlauBERT test 2020-01-30 10:04:18 -05:00
Hang Le
f0a4fc6cd6 Add Flaubert 2020-01-30 10:04:18 -05:00
Peter Izsak
a5381495e6 Added classifier dropout rate in ALBERT 2020-01-30 09:52:34 -05:00
Bram Vanroy
83446a88d9 Use _pad_token of pad_token_id 
Requesting pad_token_id would cause an error message when it is None. Use private _pad_token instead.
 2020-01-29 17:44:58 -05:00
BramVanroy
9fde13a3ac Add check to verify existence of pad_token_id 
In batch_encode_plus we have to ensure that the tokenizer has a pad_token_id so that, when padding, no None values are added as padding. That would happen with gpt2, openai, transfoxl.

closes https://github.com/huggingface/transformers/issues/2640
 2020-01-29 17:44:58 -05:00
Lysandre
e63a81dd25 Style 2020-01-29 16:29:20 -05:00
Lysandre
217349016a Copy object instead of passing the reference 2020-01-29 16:15:39 -05:00
Jared Nielsen
adb8c93134 Remove lines causing a KeyError 2020-01-29 14:01:16 -05:00
Lysandre
c69b082601 Update documentation 2020-01-29 12:06:13 -05:00
Julien Plu
ca1d66734d Apply quality and style requirements once again 2020-01-29 12:06:13 -05:00
Julien Plu
5e3c72842d bugfix on model name 2020-01-29 12:06:13 -05:00
Julien Plu
0731fa1587 Apply quality and style requirements 2020-01-29 12:06:13 -05:00
Julien Plu
a3998e76ae Add TF2 CamemBERT model 2020-01-29 12:06:13 -05:00
Lysandre
b5625f131d Style 2020-01-29 11:47:49 -05:00
Lysandre
44a5b4bbe7 Update documentation 2020-01-29 11:47:49 -05:00
Julien Plu
7fc628d98e Apply style 2020-01-29 11:47:49 -05:00
Julien Plu
64ca855617 Add TF2 XLM-RoBERTa model 2020-01-29 11:47:49 -05:00
BramVanroy
9d87eafd11 Streamlining 
- mostly stylistic streamlining
- removed 'additional context' sections. They seem to be rarely used and might cause confusion. If more details are needed, users can add them to the 'details' section
 2020-01-28 10:41:10 -05:00
BramVanroy
a3b3638f6f phrasing 2020-01-28 10:41:10 -05:00
BramVanroy
c96ca70f25 Update ---new-benchmark.md 2020-01-28 10:41:10 -05:00
BramVanroy
7b5eda32bb Update --new-model-addition.md 
Motivate users to @-tag authors of models to increase visibility and expand the community
 2020-01-28 10:41:10 -05:00
BramVanroy
c63d91dd1c Update bug-report.md 
- change references to pytorch-transformers to transformers
- link to code formatting guidelines
 2020-01-28 10:41:10 -05:00
BramVanroy
b2907cd06e Update feature-request.md 
- add 'your contribution' section
- add code formatting link to 'additional context'
 2020-01-28 10:41:10 -05:00
BramVanroy
2fec88ee02 Update question-help.md 
Prefer that general questions are asked on Stack Overflow
 2020-01-28 10:41:10 -05:00
BramVanroy
7e03d2bd7c update migration guide 
Streamlines usages of pytorch-transformers and pytorch-pretrained-bert. Add link to the README for the migration guide.
 2020-01-28 10:41:10 -05:00
Lysandre
335dd5e68a Default save steps 50 to 500 in all scripts 2020-01-28 09:42:11 -05:00
Lysandre
ea2600bd5f Absolute definitive HeisenDistilBug solve 
cc @julien-c @thomwolf
 2020-01-27 21:58:36 -05:00
Wietse de Vries
5c3d441ee1 Fix formatting 2020-01-27 21:00:34 -05:00
Wietse de Vries
f5a236c3ca Add Dutch pre-trained BERT model 2020-01-27 21:00:34 -05:00