riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-16 19:18:24 +06:00
Code Issues Actions 38 Packages Projects Releases Wiki Activity
5,115 Commits 872 Branches 227 Tags 2 GiB
49e9be0639
Commit Graph

125 Commits

Author SHA1 Message Date
Sam Shleifer
f0fc0aea6b
pegasus.rst: fix expected output (#7017) 2020-09-08 13:29:16 -04:00
Sylvain Gugger
d155b38d6e
Funnel transformer (#6908) 
* Initial model

* Fix upsampling

* Add special cls token id and test

* Formatting

* Test and fist FunnelTokenizerFast

* Common tests

* Fix the check_repo script and document Funnel

* Doc fixes

* Add all models

* Write doc

* Fix test

* Initial model

* Fix upsampling

* Add special cls token id and test

* Formatting

* Test and fist FunnelTokenizerFast

* Common tests

* Fix the check_repo script and document Funnel

* Doc fixes

* Add all models

* Write doc

* Fix test

* Fix copyright

* Forgot some layers can be repeated

* Apply suggestions from code review

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Update src/transformers/modeling_funnel.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Address review comments

* Update src/transformers/modeling_funnel.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Address review comments

* Update src/transformers/modeling_funnel.py

Co-authored-by: Sam Shleifer <sshleifer@gmail.com>

* Slow integration test

* Make small integration test

* Formatting

* Add checkpoint and separate classification head

* Formatting

* Expand list, fix link and add in pretrained models

* Styling

* Add the model in all summaries

* Typo fixes

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Sam Shleifer <sshleifer@gmail.com>
 2020-09-08 08:08:08 -04:00
Antonio V Mendoza
ea2c6f1afc
Adding the LXMERT pretraining model (MultiModal languageXvision) to HuggingFace's suite of models (#5793) 
* added template files for LXMERT and competed the configuration_lxmert.py

* added modeling, tokization, testing, and finishing touched for lxmert [yet to be tested]

* added model card for lxmert

* cleaning up lxmert code

* Update src/transformers/modeling_lxmert.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Update src/transformers/modeling_tf_lxmert.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Update src/transformers/modeling_tf_lxmert.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Update src/transformers/modeling_lxmert.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* tested torch lxmert, changed documtention, updated outputs, and other small fixes

* Update src/transformers/convert_pytorch_checkpoint_to_tf2.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Update src/transformers/convert_pytorch_checkpoint_to_tf2.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Update src/transformers/convert_pytorch_checkpoint_to_tf2.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* renaming, other small issues, did not change TF code in this commit

* added lxmert question answering model in pytorch

* added capability to edit number of qa labels for lxmert

* made answer optional for lxmert question answering

* add option to return hidden_states for lxmert

* changed default qa labels for lxmert

* changed config archive path

* squshing 3 commits: merged UI + testing improvments + more UI and testing

* changed some variable names for lxmert

* TF LXMERT

* Various fixes to LXMERT

* Final touches to LXMERT

* AutoTokenizer order

* Add LXMERT to index.rst and README.md

* Merge commit test fixes + Style update

* TensorFlow 2.3.0 sequential model changes variable names

Remove inherited test

* Update src/transformers/modeling_tf_pytorch_utils.py

* Update docs/source/model_doc/lxmert.rst

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update docs/source/model_doc/lxmert.rst

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/modeling_tf_lxmert.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* added suggestions

* Fixes

* Final fixes for TF model

* Fix docs

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
Co-authored-by: Lysandre <lysandre.debut@reseau.eseo.fr>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2020-09-03 04:02:25 -04:00
Patrick von Platen
4d1a3ffde8
[EncoderDecoder] Add xlm-roberta to encoder decoder (#6878) 
* finish xlm-roberta

* finish docs

* expose XLMRobertaForCausalLM
 2020-09-01 21:56:39 +02:00
Patrick von Platen
afc4ece462
[Generate] Facilitate PyTorch generate using ModelOutputs (#6735) 
* fix generate for GPT2 Double Head

* fix gpt2 double head model

* fix  bart / t5

* also add for no beam search

* fix no beam search

* fix encoder decoder

* simplify t5

* simplify t5

* fix t5 tests

* fix BART

* fix transfo-xl

* fix conflict

* integrating sylvains and sams comments

* fix tf past_decoder_key_values

* fix enc dec test
 2020-09-01 12:38:25 +02:00
Sam Shleifer
0ebc9699fa
[fixdoc] Add import to pegasus usage doc (#6698) 2020-08-24 15:54:57 -04:00
Stas Bekman
912a21ec78
remove BartForConditionalGeneration.generate (#6659) 
As suggested here: https://github.com/huggingface/transformers/issues/6651#issuecomment-678594233
this removes generic `generate` doc with examples not-relevant to bart.
 2020-08-25 00:42:34 +08:00
Suraj Patil
d0e42a7bed
CamembertForCausalLM (#6577) 
* added CamembertForCausalLM

* add in __init__ and auto model

* style

* doc
 2020-08-21 13:52:54 +02:00
Suraj Patil
fb6844aff5
[Pegasus Doc] minor typo (#6579) 
Minor typo correction
@sshleifer
 2020-08-18 12:47:47 -04:00
Sam Shleifer
12d7624199
[marian] converter supports models from new Tatoeba project (#6342) 2020-08-17 23:55:42 -04:00
Suraj Patil
c9564f5343
[Doc] add more MBart and other doc (#6490) 
* add mbart example

* add Pegasus and MBart in readme

* typo

* add MBart in Pretrained models

* add pre-proc doc

* add DPR in readme

* fix indent

* doc fix
 2020-08-17 12:30:26 -04:00
Patrick von Platen
36010cb1e2
fix pegasus doc (#6533) 2020-08-17 12:24:43 +02:00
Suraj Patil
680f1337c3
MBartForConditionalGeneration (#6441) 
* add MBartForConditionalGeneration

* style

* rebase and fixes

* add mbart test in TEST_FILES_WITH_NO_COMMON_TESTS

* fix docs

* don't ignore mbart

* doc

* fix mbart fairseq link

* put mbart before bart

* apply doc suggestions
 2020-08-14 03:21:16 -04:00
Patrick von Platen
0735def8e1
[EncoderDecoder] Add encoder-decoder for roberta/ vanilla longformer (#6411) 
* add encoder-decoder for roberta

* fix headmask

* apply Sylvains suggestions

* fix typo

* Apply suggestions from code review
 2020-08-12 18:23:30 +02:00
Sylvain Gugger
a8db954cda
Activate check on the CI (#6427) 
* Activate check on the CI

* Fix repo inconsistencies

* Don't document too much
 2020-08-12 08:42:14 -04:00
Sylvain Gugger
34fabe1697
Move prediction_loss_only to TrainingArguments (#6426) 2020-08-12 08:03:45 -04:00
Sam Shleifer
be1520d3a3
rename prepare_translation_batch -> prepare_seq2seq_batch (#6103) 2020-08-11 15:57:07 -04:00
Sam Shleifer
66fa8ceaea
PegasusForConditionalGeneration (torch version) (#6340) 
Co-authored-by: Jingqing  Zhang <jingqing.zhang15@imperial.ac.uk>
 2020-08-11 14:31:23 -04:00
Patrick von Platen
00bb0b25ed
TF Longformer (#5764) 
* improve names and tests longformer

* more and better tests for longformer

* add first tf test

* finalize tf basic op functions

* fix merge

* tf shape test passes

* narrow down discrepancies

* make longformer local attn tf work

* correct tf longformer

* add first global attn function

* add more global longformer func

* advance tf longformer

* finish global attn

* upload big model

* finish all tests

* correct false any statement

* fix common tests

* make all tests pass except keras save load

* fix some tests

* fix torch test import

* finish tests

* fix test

* fix torch tf tests

* add docs

* finish docs

* Update src/transformers/modeling_longformer.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Update src/transformers/modeling_tf_longformer.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* apply Lysandres suggestions

* reverse to assert statement because function will fail otherwise

* applying sylvains recommendations

* Update src/transformers/modeling_longformer.py

Co-authored-by: Sam Shleifer <sshleifer@gmail.com>

* Update src/transformers/modeling_tf_longformer.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Sam Shleifer <sshleifer@gmail.com>
 2020-08-10 23:25:06 +02:00
Sylvain Gugger
6ba540b747
Add a script to check all models are tested and documented (#6298) 
* Add a script to check all models are tested and documented

* Apply suggestions from code review

Co-authored-by: Kevin Canwen Xu <canwenxu@126.com>

* Address comments

Co-authored-by: Kevin Canwen Xu <canwenxu@126.com>
 2020-08-07 09:18:37 -04:00
Sylvain Gugger
c67d1a0259
Tf model outputs (#6247) 
* TF outputs and test on BERT

* Albert to DistilBert

* All remaining TF models except T5

* Documentation

* One file forgotten

* TF outputs and test on BERT

* Albert to DistilBert

* All remaining TF models except T5

* Documentation

* One file forgotten

* Add new models and fix issues

* Quality improvements

* Add T5

* A bit of cleanup

* Fix for slow tests

* Style
 2020-08-05 11:34:39 -04:00
Kevin Canwen Xu
3c289fb38c
Remove outdated BERT tips (#6217) 
* Remove out-dated BERT tips

* Update modeling_outputs.py

* Update bert.rst

* Update bert.rst
 2020-08-04 01:17:56 +08:00
Faiaz Rahman
a39dfe4fb1
Fixed typo in Longformer (#6180) 2020-08-01 18:20:48 +08:00
Oren Amsalem
d24ea708d7
Actually the extra_id are from 0-99 and not from 1-100 (#5967) 
a = tokenizer.encode("we got a <extra_id_99>", return_tensors='pt',add_special_tokens=True)
print(a)
>tensor([[   62,   530,     3,     9, 32000]])
a = tokenizer.encode("we got a <extra_id_100>", return_tensors='pt',add_special_tokens=True)
print(a)
>tensor([[   62,   530,     3,     9,     3,     2, 25666,   834,    23,    26,
           834,  2915,  3155]])
 2020-07-30 06:13:29 -04:00
Sylvain Gugger
a20969170b
Add AlbertForPretraining to doc (#5914) 2020-07-20 17:53:21 -04:00
Stas Bekman
45addfe96d
FlaubertForTokenClassification (#5644) 
* implement FlaubertForTokenClassification as a subclass of XLMForTokenClassification

* fix mapping order

* add the doc

* add common tests
 2020-07-13 14:59:53 -04:00
Sylvain Gugger
7fad617dc1
Document model outputs (#5673) 
* Document model outputs

* Update docs/source/main_classes/output.rst

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
 2020-07-10 17:31:02 -04:00
Sam Shleifer
353b8f1e7a
Add mbart-large-cc25, support translation finetuning (#5129) 
improve unittests for finetuning, especially w.r.t testing frozen parameters
fix freeze_embeds for T5
add streamlit setup.cfg
 2020-07-07 13:23:01 -04:00
Quentin Lhoest
fbd8792195
Add DPR model (#5279) 
* beginning of dpr modeling

* wip

* implement forward

* remove biencoder + better init weights

* export dpr model to embed model for nlp lib

* add new api

* remove old code

* make style

* fix dumb typo

* don't load bert weights

* docs

* docs

* style

* move the `k` parameter

* fix init_weights

* add pretrained configs

* minor

* update config names

* style

* better config

* style

* clean code based on PR comments

* change Dpr to DPR

* fix config

* switch encoder config to a dict

* style

* inheritance -> composition

* add messages in assert startements

* add dpr reader tokenizer

* one tokenizer per model

* fix base_model_prefix

* fix imports

* typo

* add convert script

* docs

* change tokenizers conf names

* style

* change tokenizers conf names

* minor

* minor

* fix wrong names

* minor

* remove unused convert functions

* rename convert script

* use return_tensors in tokenizers

* remove n_questions dim

* move generate logic to tokenizer

* style

* add docs

* docs

* quality

* docs

* add tests

* style

* add tokenization tests

* DPR full tests

* Stay true to the attention mask building

* update docs

* missing param in bert input docs

* docs

* style

Co-authored-by: Lysandre <lysandre.debut@reseau.eseo.fr>
 2020-07-07 08:56:12 -04:00
Patrick von Platen
d16e36c7e5
[Reformer] Add Masked LM Reformer (#5426) 
* fix conflicts

* fix

* happy rebasing
 2020-07-01 22:43:18 +02:00
Patrick von Platen
fe81f7d12c
finish reformer qa head (#5433) 2020-07-01 12:27:14 -04:00
Sylvain Gugger
1262495a91
Add TF auto model to the docs + fix sphinx warnings (#5187) 2020-06-22 14:43:52 -04:00
Vasily Shamporov
9a3f91088c
Add MobileBert (#4901) 
* Add MobileBert

* Quality + Conversion script

* style

* Update src/transformers/modeling_mobilebert.py

* Links to S3

* Style

* TFMobileBert

Slight fixes to the pytorch MobileBert
Style

* MobileBertForMaskedLM (PT + TF)

* MobileBertForNextSentencePrediction (PT + TF)

* MobileFor{MultipleChoice, TokenClassification} (PT + TF)


ss

* Tests + Auto

* Doc

* Tests

* Addressing @sgugger's comments

* Adressing @patrickvonplaten's comments

* Style

* Style

* Integration test

* style

* Model card

Co-authored-by: Lysandre <lysandre.debut@reseau.eseo.fr>
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
 2020-06-19 16:38:36 -04:00
Suraj Patil
18177a1a60
lm_labels => labels (#5080) 2020-06-18 09:16:29 +02:00
Sylvain Gugger
204ebc25e6
Update installation page and add contributing to the doc (#5084) 
* Update installation page and add contributing to the doc

* Remove mention of symlinks
 2020-06-17 14:01:10 -04:00
Sylvain Gugger
011cc0be51
Fix all sphynx warnings (#5068) 2020-06-16 16:50:02 -04:00
Yacine Jernite
49c5202522
Eli5 examples (#4968) 
* add eli5 examples

* add dense query script

* query_di

* merging

* merging

* add_utils

* adds nearest neighbor wikipedia

* batch queries

* training_retriever

* new notebooks

* moved retriever traiing script

* finished wiki40b

* max_len_fix

* train_s2s

* retriever_batch_checkpointing

* cleanup

* merge

* dim_fix

* fix_indexer

* fix_wiki40b_snippets

* fix_embed_for_r

* fp32 index

* fix_sparse_q

* joint_training

* remove obsolete datasets

* add_passage_nn_results

* add_passage_nn_results

* add_batch_nn

* add_batch_nn

* add_data_scripts

* notebook

* notebook

* notebook

* fix_multi_gpu

* add_app

* full_caching

* full_caching

* notebook

* sparse_done

* images

* notebook

* add_image_gif

* with_Gif

* add_contr_image

* notebook

* notebook

* notebook

* train_functions

* notebook

* min_retrieval_length

* pandas_option

* notebook

* min_retrieval_length

* notebook

* notebook

* eval_Retriever

* notebook

* images

* notebook

* add_example

* add_example

* notebook

* fireworks

* notebook

* notebook

* joe's notebook comments

* app_update

* notebook

* notebook_link

* captions

* notebook

* assing RetriBert model

* add RetriBert to Auto

* change AutoLMHead to AutoSeq2Seq

* notebook downloads from hf models

* style_black

* style_black

* app_update

* app_update

* fix_app_update

* style

* style

* isort

* Delete WikiELI5training.ipynb

* Delete evaluate_eli5.py

* Delete WikiELI5explore.ipynb

* Delete ExploreWikiELI5Support.html

* Delete explainlikeimfive.py

* Delete wiki_snippets.py

* children before parent

* children before parent

* style_black

* style_black_only

* isort

* isort_new

* Update src/transformers/modeling_retribert.py

Co-authored-by: Julien Chaumond <chaumond@gmail.com>

* typo fixes

* app_without_asset

* cleanup

* Delete ELI5animation.gif

* Delete ELI5contrastive.svg

* Delete ELI5wiki_index.svg

* Delete choco_bis.svg

* Delete fireworks.gif

* Delete huggingface_logo.jpg

* Delete huggingface_logo.svg

* Delete Long_Form_Question_Answering_with_ELI5_and_Wikipedia.ipynb

* Delete eli5_app.py

* Delete eli5_utils.py

* readme

* Update README.md

* unused imports

* moved_info

* default_beam

* ftuned model

* disclaimer

* Update src/transformers/modeling_retribert.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* black

* add_doc

* names

* isort_Examples

* isort_Examples

* Add doc to index

Co-authored-by: Julien Chaumond <chaumond@gmail.com>
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
Co-authored-by: Lysandre <lysandre.debut@reseau.eseo.fr>
 2020-06-16 16:36:58 -04:00
Sylvain Gugger
f9f8a5312e
Add DistilBertForMultipleChoice (#5032) 
* Add `DistilBertForMultipleChoice`
 2020-06-15 18:31:41 -04:00
Suraj Patil
e93ccb3290
BartForQuestionAnswering (#4908) 2020-06-12 15:47:57 -04:00
Sylvain Gugger
538531cde5
Add AlbertForMultipleChoice (#4959) 
* Add AlbertForMultipleChoice

* Make up to date and add all models to common tests
 2020-06-12 14:20:19 -04:00
Suraj Patil
ef2dcdccaa
ElectraForQuestionAnswering (#4913) 
* ElectraForQuestionAnswering

* udate __init__

* add test for electra qa model

* add ElectraForQuestionAnswering in auto models

* add ElectraForQuestionAnswering in all_model_classes

* fix outputs, input_ids defaults to None

* add ElectraForQuestionAnswering in docs

* remove commented line
 2020-06-10 15:17:52 -04:00
Sylvain Gugger
41a1d27cde
Add XLMRobertaForQuestionAnswering (#4855) 
* Add XLMRobertaForQuestionAnswering

* Formatting

* Make test happy
 2020-06-08 21:22:37 -04:00
Sylvain Gugger
37be3786cf
Clean documentation (#4849) 
* Clean documentation
 2020-06-08 11:28:19 -04:00
Julien Chaumond
b42586ea56
Fix CI after killing archive maps (#4724) 
* 🐛 Fix model ids for BART and Flaubert
 2020-06-02 10:21:09 -04:00
Patrick von Platen
56ee2560be
[Longformer] Better handling of global attention mask vs local attention mask (#4672) 
* better api

* improve automatic setting of global attention mask

* fix longformer bug

* fix global attention mask in test

* fix global attn mask flatten

* fix slow tests

* update docstring

* update docs and make more robust

* improve attention mask
 2020-05-29 17:58:42 +02:00
Patrick von Platen
9c17256447
[Longformer] Multiple choice for longformer (#4645) 
* add multiple choice for longformer

* add models to docs

* adapt docstring

* add test to longformer

* add longformer for mc in init and modeling auto

* fix tests
 2020-05-29 13:46:08 +02:00
Patrick von Platen
c589eae2b8
[Longformer For Question Answering] Conversion script, doc, small fixes (#4593) 
* add new longformer for question answering model

* add new config as well

* fix links

* fix links part 2
 2020-05-26 14:58:47 +02:00
Patrick von Platen
3e3e552125
[Reformer] fix reformer num buckets (#4564) 
* fix reformer num buckets

* fix

* adapt docs

* set num buckets in config
 2020-05-25 16:04:45 -04:00
Alexander Measure
95a26fcf2d
link to paper was broken (#4526) 
changed from https://https://arxiv.org/abs/2001.04451.pdf to https://arxiv.org/abs/2001.04451.pdf
 2020-05-22 15:17:09 -04:00
Patrick von Platen
48c3a70b4e
[Longformer] Docs and clean API (#4464) 
* add longformer docs

* improve docs
 2020-05-19 21:52:36 +02:00