riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-27 00:09:00 +06:00
Code Issues Actions 27 Packages Projects Releases Wiki Activity
2,207 Commits 888 Branches 231 Tags 2.1 GiB
c110c41fdb
Commit Graph

2207 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Louis MARTIN
3e20c2e871 Update demo_camembert.py with new classes 2019-11-16 00:11:07 -05:00
Louis MARTIN
f12e4d8da7 Move demo_camembert.py to examples/contrib 2019-11-16 00:11:07 -05:00
Louis MARTIN
fb6c70a91d Update tokenization_camembert.py with urls 2019-11-16 00:11:07 -05:00
Louis MARTIN
e44b939e71 Add configuration_camembert.py and modeling_camembert.py 2019-11-16 00:11:07 -05:00
Louis MARTIN
6e72fd094c Add demo_camembert.py 2019-11-16 00:11:07 -05:00
Louis MARTIN
14b3aa3b3c Add tokenization_camembert.py 2019-11-16 00:11:07 -05:00
Thomas Wolf
74ce8de7d8
Merge pull request #1792 from stefan-it/distilbert-for-token-classification 
DistilBERT for token classification
 2019-11-14 22:47:53 +01:00
Thomas Wolf
05db5bc1af
added small comparison between BERT, RoBERTa and DistilBERT 2019-11-14 22:40:22 +01:00
Thomas Wolf
9629e2c676
Merge pull request #1804 from ronakice/master 
fix multi-gpu eval in torch examples
 2019-11-14 22:24:05 +01:00
Thomas Wolf
5b322a36db
Merge pull request #1811 from huggingface/special-tokens 
Fix special tokens addition in decoder #1807
 2019-11-14 22:17:24 +01:00
Thomas Wolf
1a237d7f42
Merge pull request #1831 from iedmrc/gpt2-tokenization-sum-func-replacement 
sum() is replaced by itertools.chain.from_iterable()
 2019-11-14 22:11:54 +01:00
Thomas Wolf
df99f8c5a1
Merge pull request #1832 from huggingface/memory-leak-schedulers 
replace LambdaLR scheduler wrappers by function
 2019-11-14 22:10:31 +01:00
Thomas Wolf
0be9ae7b3e
Merge pull request #1833 from huggingface/max-length-warning 
Token indices sequence length is longer than the specified maximum sequence length for this model
 2019-11-14 22:04:49 +01:00
Lysandre
be7f2aacce [CI][DOC] Don't rebuild if folder exists - Correct directory. 2019-11-14 14:54:44 -05:00
Lysandre
8f8d69716a [CI][DOC] Don't rebuild if folder exists. 2019-11-14 14:48:21 -05:00
Rémi Louf
2276bf69b7 update the examples, docs and template 2019-11-14 20:38:02 +01:00
Lysandre
d7929899da Specify checkpoint in saved file for run_lm_finetuning.py 2019-11-14 10:49:00 -05:00
Lysandre
a67e747889 Reorganized max_len warning 2019-11-14 10:30:22 -05:00
Lysandre
e18f786cd5 Quickstart example showcasing past 2019-11-14 10:06:00 -05:00
Rémi Louf
022525b003 replace LambdaLR scheduler wrappers by function 
Custom schedulers are currently initiated by wrapping Pytorch's LambdaLR
class and passing a method of the wrapping class to the __init__
function of LambdaLR. This approach is not appropriate for several
reasons:

1. one does not need to define a class when it only defines a
__init__() method;
2. instantiating the parent class by passing a method of the child class
creates a cyclical reference which leads to memory leaks. See issues #1742 and #1134.

In this commit we replace the wrapper classes with functions that
instantiate `LambdaLR` with a custom learning rate function. We use a
closure to specify the parameter of the latter. We also do a bit of
renaming within the function to explicit the behaviour and removed
docstrings that were subsequently not necessary.
 2019-11-14 15:39:08 +01:00
İbrahim Ethem Demirci
7627dde1f8 sum() is the leanest method to flatten a string list, so it's been replaced by itertools.chain.from_iterable() 2019-11-14 17:06:15 +03:00
Lysandre
74d0bcb6ff Fix special tokens addition in decoder 2019-11-12 15:27:57 -05:00
Julien Chaumond
155c782a2c [inputs_embeds] All TF models + tests 2019-11-12 11:29:21 -05:00
Julien Chaumond
2aef2f0bbc [common attributes] Fix previous commit for transfo-xl 2019-11-12 11:29:21 -05:00
Julien Chaumond
2f17464266 [common attributes] Slightly sharper test coverage 2019-11-12 11:29:21 -05:00
Julien Chaumond
9d2398fd99 Ooopsie 2019-11-12 11:29:21 -05:00
Julien Chaumond
70d97ddd60 [TF models] Common attributes as per #1721 2019-11-12 11:29:21 -05:00
Julien Chaumond
872403be1c This is not a @property after all 2019-11-12 11:29:21 -05:00
Julien Chaumond
dd6b2e05e1 whitespace 2019-11-12 11:29:21 -05:00
Lysandre
d409aca326 Clarify the use of past in GPT2 and CTRL 2019-11-12 10:59:37 -05:00
ronakice
2e31176557 fix multi-gpu eval 2019-11-12 05:55:11 -05:00
thomwolf
8aba81a0b6 fix #1789 2019-11-12 08:52:43 +01:00
Stefan Schweter
94e55253ae tests: add test case for DistilBertForTokenClassification implementation 2019-11-11 16:20:15 +01:00
Stefan Schweter
2b07b9e5ee examples: add DistilBert support for NER fine-tuning 2019-11-11 16:19:34 +01:00
Stefan Schweter
1806eabf59 module: add DistilBertForTokenClassification import 2019-11-11 16:18:48 +01:00
Stefan Schweter
1c7253cc5f modeling: add DistilBertForTokenClassification implementation 2019-11-11 16:18:16 +01:00
Lysandre
b5d330d118 Fix #1784 2019-11-11 10:15:14 -05:00
Adrian Bauer
7a9aae1044 Fix run_bertology.py 
Make imports and args.overwrite_cache match run_glue.py
 2019-11-08 16:28:40 -05:00
Julien Chaumond
1c542df7e5 Add RoBERTa-based GPT-2 Output Detector from OpenAI 
converted from https://github.com/openai/gpt-2-output-dataset/tree/master/detector

Co-Authored-By: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
Co-Authored-By: Jong Wook Kim <jongwook@nyu.edu>
Co-Authored-By: Jeff Wu <wuthefwasthat@gmail.com>
 2019-11-06 16:26:31 -05:00
Julien Chaumond
2f3a421018 Fix other PyTorch models 2019-11-06 14:03:47 -05:00
Julien Chaumond
d5319793c4 Fix BERT 2019-11-06 14:03:47 -05:00
Julien Chaumond
27e015bd54 [tests] Flag to test on cuda 2019-11-06 14:03:47 -05:00
Julien Chaumond
13d9135fa5 [tests] get rid of warning 
cf. https://docs.pytest.org/en/latest/example/simple.html
 2019-11-06 14:03:47 -05:00
Julien Chaumond
f88c104d8f [run_tf_glue] Add comment for context 2019-11-05 19:56:43 -05:00
Julien Chaumond
30968d70af misc doc 2019-11-05 19:06:12 -05:00
Dom Hudson
de890ae67d Updating docblocks in optimizers.py 2019-11-05 17:31:29 -05:00
Lysandre
d7d36181fd GPT-2 XL 2019-11-05 13:31:58 -05:00
Julien Chaumond
7daacf00df
Merge pull request #1695 from huggingface/models_inputs_embeds 
model forwards can take an inputs_embeds param
 2019-11-05 09:55:28 -05:00
Clement
a44f112fb9
add authors for models 2019-11-05 08:48:26 -05:00
Thomas Wolf
e99071f105
Merge pull request #1734 from orena1/patch-1 
add progress bar to convert_examples_to_features
 2019-11-05 11:34:20 +01:00