riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-13 17:48:22 +06:00
Code Issues Actions 14 Packages Projects Releases Wiki Activity
4,424 Commits 864 Branches 226 Tags 1.9 GiB
7ecff0ccbb
Commit Graph

297 Commits

Author SHA1 Message Date
Lysandre Debut
bb7c468520
Documentation (#2989) 
* All Tokenizers

BertTokenizer + few fixes
RobertaTokenizer
OpenAIGPTTokenizer + Fixes
GPT2Tokenizer + fixes
TransfoXLTokenizer
Correct rst for TransformerXL
XLMTokenizer + fixes
XLNet Tokenizer + Style
DistilBERT + Fix XLNet RST
CTRLTokenizer
CamemBERT Tokenizer
FlaubertTokenizer
XLMRobertaTokenizer
cleanup

* cleanup
 2020-02-25 18:43:36 -05:00
Lysandre Debut
65e7c90a77
Adding usage examples for common tasks (#2850) 
* Usage: Sequence Classification & Question Answering

* Pipeline example

* Language modeling

* TensorFlow code for Sequence classification

* Custom TF/PT toggler in docs

* QA + LM for TensorFlow

* Finish Usage for both PyTorch and TensorFlow

* Addressing Julien's comments

* More assertive

* cleanup

* Favicon
- added favicon option in conf.py along with the favicon image
- udpated 🤗 logo. slightly smaller and should appear more consistent across editing programs (no more tongue on the outside of the mouth)

Co-authored-by: joshchagani <joshua@joshuachagani.com>
 2020-02-25 13:48:24 -05:00
Lysandre
f9ec5ca90b Release: v2.5.1 2020-02-24 18:22:54 -05:00
Sam Shleifer
53ce3854a1
New BartModel (#2745) 
* Results same as fairseq
* Wrote a ton of tests
* Struggled with api signatures
* added some docs
 2020-02-20 18:11:13 -05:00
Lysandre
fb560dcb07 Release: v2.5.0 
Welcome Rust Tokenizers
 2020-02-19 11:46:19 -05:00
Lysandre
fd639e5be3 Correct quickstart example when using the past 2020-02-10 11:25:56 -05:00
Lysandre
dd28830327 Update RoBERTa tips 2020-02-07 16:42:35 -05:00
Lysandre
db97930122 Update XLM-R tips 2020-02-07 16:42:35 -05:00
VictorSanh
ee5a6856ca distilbert-base-cased weights + Readmes + omissions 2020-02-07 15:28:13 -05:00
Julien Chaumond
42f08e596f [examples] rename run_lm_finetuning to run_language_modeling 2020-02-07 09:15:28 -05:00
Julien Chaumond
7748cbbe7d Oopsie 2020-02-06 15:30:02 -05:00
Julien Chaumond
432c12521e [docs] Add menu w/ links to other pages on hf.co 2020-02-06 15:30:02 -05:00
Julien Chaumond
eae8ee0389 [doc] model sharing: mention README.md + tweaks 
cc @lysandrejik @thomwolf
 2020-02-05 14:20:03 -05:00
Lysandre
9c67196b83 Update quickstart 2020-02-04 11:11:37 -05:00
Lysandre
d426b58b9e Patch: v2.4.1 2020-01-31 14:55:33 -05:00
Lysandre
6664ea943d Release: v2.4.0 2020-01-31 09:40:32 -05:00
Hang Le
b43cb09aaa Add layerdrop 2020-01-30 12:05:01 -05:00
Lysandre
93dccf527b Pretrained models 2020-01-30 10:04:18 -05:00
Lysandre
73306d028b FlauBERT documentation 2020-01-30 10:04:18 -05:00
Lysandre
c69b082601 Update documentation 2020-01-29 12:06:13 -05:00
Lysandre
44a5b4bbe7 Update documentation 2020-01-29 11:47:49 -05:00
Wietse de Vries
f5a236c3ca Add Dutch pre-trained BERT model 2020-01-27 21:00:34 -05:00
thomwolf
e0849a66ac adding in the doc 2020-01-27 14:27:07 -05:00
Lysandre
983fef469c AutoModels doc 2020-01-24 16:37:30 -05:00
Lysandre
24d5ad1dcc Run the examples in slow 2020-01-23 09:38:45 -05:00
Lysandre
9ddf60b694 Tips + whitespaces 2020-01-23 09:38:45 -05:00
Lysandre
0e9899f451 Fixes 2020-01-23 09:38:45 -05:00
Lysandre
7511f3dd89 PyTorch CTRL + Style 2020-01-23 09:38:45 -05:00
Lysandre
980211a63a XLM-RoBERTa 2020-01-23 09:38:45 -05:00
Lysandre
db1a7f27a1 PyTorch DistilBERT 2020-01-23 09:38:45 -05:00
Lysandre
b28020f590 TF RoBERTa 2020-01-23 09:38:45 -05:00
Lysandre
3e1bc27e1b Pytorch RoBERTa 2020-01-23 09:38:45 -05:00
Lysandre
f44ff574d3 Camembert 2020-01-23 09:38:45 -05:00
Lysandre
ccebcae75f PyTorch XLM 2020-01-23 09:38:45 -05:00
Lysandre
cd656fb21a PyTorch XLNet 2020-01-23 09:38:45 -05:00
Lysandre
98edad418e PyTorch Transformer-XL 2020-01-23 09:38:45 -05:00
Lysandre
850795c487 Pytorch GPT 2020-01-23 09:38:45 -05:00
Lysandre
1487b840d3 TF GPT2 2020-01-23 09:38:45 -05:00
Lysandre
bd0d3fd76e GPT-2 PyTorch models + better tips for BERT 2020-01-23 09:38:45 -05:00
Lysandre
cd77c750c5 BERT PyTorch models 2020-01-23 09:38:45 -05:00
Lysandre
3922a2497e TF ALBERT + TF Utilities + Fix warnings 2020-01-23 09:38:45 -05:00
Lysandre
00df3d4de0 ALBERT Modeling + required changes to utilities 2020-01-23 09:38:45 -05:00
Lysandre
632675ea88 Can test examples spread over multiple blocks 2020-01-23 09:38:45 -05:00
Lysandre
9bab9b83d2 Glossary 2020-01-23 09:38:45 -05:00
Julien Chaumond
119dc50e2a Doc tweak on model sharing 2020-01-22 22:40:38 -05:00
Lysandre
387217bd3e Added example usage 2020-01-14 14:09:09 +01:00
Lysandre
7d1bb7f256 Add missing XLNet and XLM models 2020-01-14 14:09:09 +01:00
Lysandre Debut
632682726f Updated Configurations 2020-01-14 14:09:09 +01:00
alberduris
81d6841b4b GPU text generation: mMoved the encoded_prompt to correct device 2020-01-06 15:11:12 +01:00
alberduris
dd4df80f0b Moved the encoded_prompts to correct device 2020-01-06 15:11:12 +01:00
Morgan Funtowicz
80faf22b4a Updating documentation for converting tensorflow model to reflect the new cli convert format. 
Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
 2020-01-04 13:41:18 +01:00
Julien Chaumond
9b2badf3c9 [cli] Update doc 2019-12-27 22:54:29 -05:00
Aymeric Augustin
a8d34e534e Remove [--editable] in install instructions. 
Use -e only in docs targeted at contributors.

If a user copy-pastes  command line with [--editable], they will hit
an error. If they don't know the --editable option, we're giving them
a choice to make before they can move forwards, but this isn't a choice
they need to make right now.
 2019-12-24 08:46:08 +01:00
Aymeric Augustin
70373a5f7c Update contribution instructions. 
Also provide shortcuts in a Makefile.
 2019-12-23 21:05:30 +01:00
Aymeric Augustin
d8e33dbd67 Fix path to source code in docs config. 
This should fix API docs, which went AWOL with yesterday's changes.
 2019-12-23 16:49:35 +01:00
Aymeric Augustin
45841eaf7b Remove references to Python 2 in documentation. 2019-12-22 18:38:56 +01:00
Aymeric Augustin
ced0a94204 Switch test files to the standard test_*.py scheme. 2019-12-22 14:15:13 +01:00
Aymeric Augustin
067395d5c5 Move tests outside of library. 2019-12-22 13:47:17 +01:00
Julien Chaumond
ac1b449cc9 [doc] move distilroberta to more appropriate place 
cc @lysandrejik
 2019-12-21 00:09:01 -05:00
Lysandre
a436574bfd Release: v2.3.0 2019-12-20 16:22:20 -05:00
Rémi Louf
4e3f745ba4 add example for Model2Model in quickstart 2019-12-20 09:12:31 -05:00
Stefan Schweter
f09d999641 docs: fix numbering 😅 2019-12-18 19:49:33 +01:00
Stefan Schweter
dd7a958fd6 docs: add XLM-RoBERTa to pretrained model list (incl. all parameters) 2019-12-18 19:45:46 +01:00
Stefan Schweter
d35405b7a3 docs: add XLM-RoBERTa to index page 2019-12-18 19:45:10 +01:00
Antti Virtanen
abc43ffbff Add pretrained model documentation for FinBERT. 2019-12-17 20:35:25 -05:00
Julien Chaumond
3f5ccb183e [doc] Clarify uploads 
cf 855ff0e91d (commitcomment-36452545)
 2019-12-16 18:20:29 -05:00
Julien Chaumond
855ff0e91d [doc] Model upload and sharing 
ping @lysandrejik @thomwolf

Is this clear enough? Anything we should add?
 2019-12-16 12:42:22 -05:00
Thomas Wolf
e92bcb7eb6
Merge pull request #1739 from huggingface/t5 
[WIP] Adding Google T5 model
 2019-12-14 09:40:43 +01:00
Lysandre
7bd11dda6f Release: v2.2.2 2019-12-13 16:45:30 -05:00
thomwolf
5c00e344c1 update model doc - swith 3B/11B to 3b/11b 2019-12-13 16:33:29 +01:00
Thomas Wolf
110394b2ba
Merge branch 'master' into t5 2019-12-13 16:03:32 +01:00
Julien Chaumond
1748fdf657 [doc] Fix rst table 2019-12-11 18:32:27 -05:00
Masatoshi Suzuki
c03c0dfd23 Add support for Japanese BERT models by cl-tohoku 2019-12-11 18:32:27 -05:00
Stefan Schweter
030faccb8d doc: fix pretrained models table 2019-12-11 12:19:21 -05:00
thomwolf
0558c9cb9b Merge branch 'master' into t5 2019-12-10 12:58:48 +01:00
Thomas Wolf
e57d00ee10
Merge pull request #1984 from huggingface/squad-refactor 
[WIP] Squad refactor
 2019-12-10 11:07:26 +01:00
Pierric Cistac
5c877fe94a
fix albert links 2019-12-09 18:53:00 -05:00
Lysandre Debut
00c4e39581
Merge branch 'master' into squad-refactor 2019-12-09 10:41:15 -05:00
Aymeric Augustin
35401fe50f Remove dependency on pytest for running tests (#2055) 
* Switch to plain unittest for skipping slow tests.

Add a RUN_SLOW environment variable for running them.

* Switch to plain unittest for PyTorch dependency.

* Switch to plain unittest for TensorFlow dependency.

* Avoid leaking open files in the test suite.

This prevents spurious warnings when running tests.

* Fix unicode warning on Python 2 when running tests.

The warning was:

    UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal

* Support running PyTorch tests on a GPU.

Reverts 27e015bd.

* Tests no longer require pytest.

* Make tests pass on cuda
 2019-12-06 13:57:38 -05:00
Thomas Wolf
5482822a2b
Merge pull request #2046 from jplu/tf2-ner-example 
Add NER TF2 example.
 2019-12-06 12:12:22 +01:00
LysandreJik
9ecd83dace Patch evaluation for impossible values + cleanup 2019-12-05 14:44:57 -05:00
VictorSanh
552c44a9b1 release distilm-bert 2019-12-05 10:14:58 -05:00
Julien Plu
9200a759d7 Add few tests on the TF optimization file with some info in the documentation. Complete the README. 2019-12-05 12:56:43 +01:00
Thomas Wolf
1f179f095f
Merge pull request #2011 from AdityaSoni19031997/patch-1 
typo fix on the docs as per Pytorch v1.1+
 2019-12-05 12:39:04 +01:00
LysandreJik
7a03519975 Documentation 2019-12-04 17:24:35 -05:00
LysandreJik
8101924a68 Patch: v2.2.1 2019-12-03 11:20:26 -05:00
Aditya Soni
c356290c8d
typo fix as per Pytorch v1.1+ 2019-12-01 14:08:14 +05:30
Stefan Schweter
8c276b9c92
Merge branch 'master' into distilbert-german 2019-11-27 18:11:49 +01:00
VictorSanh
d75d49a51d add XnliProcessor to doc 2019-11-27 11:07:22 -05:00
Lysandre
361620954a Remove TFBertForPreTraining from ALBERT doc 2019-11-27 10:11:37 -05:00
Lysandre
ce02550d50 Fix pretrained models table 2019-11-26 15:47:02 -05:00
Lysandre
cf26a0c85e Fix pretrained models table 2019-11-26 15:40:03 -05:00
Lysandre
ee4647bd5c CamemBERT & ALBERT doc 2019-11-26 15:10:51 -05:00
Lysandre
668aac45d2 Pretrained models 2019-11-26 14:52:42 -05:00
Lysandre
ae98d45991 Release: v2.2.0 2019-11-26 14:12:44 -05:00
Julien Chaumond
afaa335851 [doc] Fix assets urls 2019-11-23 11:34:45 -05:00
Stefan Schweter
e631383d4f docs: add new German distilbert model to pretrained models 2019-11-19 19:52:40 +01:00
Louis MARTIN
035fea5315 Add CamemBERT to auto files and docs 2019-11-16 00:11:07 -05:00
Thomas Wolf
df99f8c5a1
Merge pull request #1832 from huggingface/memory-leak-schedulers 
replace LambdaLR scheduler wrappers by function
 2019-11-14 22:10:31 +01:00
Rémi Louf
2276bf69b7 update the examples, docs and template 2019-11-14 20:38:02 +01:00
Lysandre
e18f786cd5 Quickstart example showcasing past 2019-11-14 10:06:00 -05:00
thomwolf
f03c0c1423 adding models in readme and auto classes 2019-11-08 11:49:46 +01:00
Julien Chaumond
1c542df7e5 Add RoBERTa-based GPT-2 Output Detector from OpenAI 
converted from https://github.com/openai/gpt-2-output-dataset/tree/master/detector

Co-Authored-By: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
Co-Authored-By: Jong Wook Kim <jongwook@nyu.edu>
Co-Authored-By: Jeff Wu <wuthefwasthat@gmail.com>
 2019-11-06 16:26:31 -05:00
Julien Chaumond
30968d70af misc doc 2019-11-05 19:06:12 -05:00
Lysandre
d7d36181fd GPT-2 XL 2019-11-05 13:31:58 -05:00
Julien Chaumond
93d2fff071 Close #1654 2019-11-01 09:47:38 -04:00
VictorSanh
8ad5c591cd [RELEASE] DistilRoBERTa 2019-10-23 10:29:47 -04:00
LysandreJik
82f6abd98a Benchmark section added to the documentation 2019-10-18 17:27:10 -04:00
Lysandre
3ddce1d74c Release: 2.1.1 2019-10-11 06:37:49 -04:00
Stefan Schweter
5f25a5f367 model: add support for new German BERT models (cased and uncased) from @dbmdz 2019-10-11 10:20:33 +02:00
LysandreJik
9c2e0a4acf Release: 2.1.0 2019-10-09 12:14:03 -04:00
LysandreJik
7fe98d8c18 Update CTRL documentation 2019-10-09 12:12:36 -04:00
LysandreJik
89f86f9661 CTRL added to the documentation 2019-10-09 12:04:06 -04:00
thomwolf
d9e60f4f0d Merge branch 'master' into pr/1383 2019-10-09 17:25:08 +02:00
thomwolf
48b438ff2a doc and conversion 2019-10-09 17:06:30 +02:00
Julien Chaumond
d688af19e5 Update link to swift-coreml-transformers 
cc @lysandrejik
 2019-10-08 16:37:52 -04:00
LysandreJik
8fcc6507ce Multilingual 2019-10-07 15:02:42 -04:00
Thomas Wolf
b3cfd97946
Merge pull request #1373 from TimYagan/fix-css 
Fixed critical css font-family issues
 2019-10-03 19:04:02 -04:00
VictorSanh
e2ae9c0b73 fix links in doc index 2019-10-03 11:42:21 -04:00
VictorSanh
c1689ac301 fix name 2019-10-03 10:56:39 -04:00
VictorSanh
4a790c40b1 update doc for distil* 2019-10-03 10:54:02 -04:00
LysandreJik
ebb32261b1 fix #1401 2019-10-02 17:52:56 -04:00
Tim Yagan
0a4ed7192e
Fixed critical css font-family issues 
Fixed critical css font-family issues to ensure compatibility with multiple webbrowsers
 2019-09-29 13:51:01 +02:00
Julien Chaumond
d8b641c839 6 -> 8 models 2019-09-27 17:22:01 -04:00
pj
4f2b6579bf
Fix some typos 2019-09-27 22:55:43 +08:00
Gabriel Luiz Freitas Almeida
d2de5b9d8c
Just some typos 2019-09-27 07:08:36 -03:00
Julien Chaumond
fc9faa8a47 [docs] Doc tweaks 
Co-Authored-By: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
 2019-09-26 18:19:51 -04:00
LysandreJik
93f0c5fc72 Repository link in the documentation 2019-09-26 11:45:00 -04:00
thomwolf
6c3b131516 typo in readme/doc 2019-09-26 16:23:28 +02:00
LysandreJik
7e957237e4 [Doc] XLM + Torch in documentation 2019-09-26 10:08:56 -04:00
LysandreJik
927904bc91 [doc] pytorch_transformers -> transformers 2019-09-26 08:47:15 -04:00
LysandreJik
294edfd83d Release version in documentation 2019-09-26 08:16:12 -04:00
LysandreJik
de5e4864cb Documentation 2019-09-26 08:04:54 -04:00
LysandreJik
8349d75773 Various small doc fixes 2019-09-26 07:45:40 -04:00
LysandreJik
fb056494e5 Example usage 2019-09-26 07:45:40 -04:00
LysandreJik
36f592cc82 Updated doc for InputExample and InputFeatures 2019-09-26 07:45:40 -04:00
LysandreJik
ad4a393e2e Changed processor documentation architecture. Added documentation for GLUE 2019-09-26 07:45:40 -04:00
LysandreJik
c4ac7a76db GLUE processors 2019-09-26 07:45:40 -04:00
LysandreJik
4acd87ff4e TF models added to documentation 2019-09-26 07:45:40 -04:00
LysandreJik
cf5c5c9e1c Documentation 2019-09-26 07:43:13 -04:00
thomwolf
f47f7f4611 add logo 2019-09-26 11:28:44 +02:00
thomwolf
31c23bd5ee [BIG] pytorch-transformers => transformers 2019-09-26 10:15:53 +02:00
thomwolf
c88f05163d fix typo in XLM models 2019-09-16 13:42:20 +02:00
LysandreJik
593c070435 Better examples 2019-09-06 12:00:12 -04:00
LysandreJik
0b52642d37 1.2.0 in docs 2019-09-04 11:03:32 -04:00
LysandreJik
7f522437bc Updated documentation for LM finetuning script 2019-09-02 13:40:25 -04:00
Julien Chaumond
2dcc5a1629 [doc] Add blurb about large-scale model downloads 
cc @n1t0 @lysandrejik @thomwolf
 2019-09-02 12:27:11 -04:00
LysandreJik
09363f2a8b Fix documentation index 2019-08-30 19:48:32 -04:00
LysandreJik
e0caab0cf0 fix link 2019-08-30 10:09:17 -04:00
LysandreJik
a600b30cc3 Fix index number in documentation 2019-08-30 10:08:14 -04:00