Patrick von Platen
afc4ece462
[Generate] Facilitate PyTorch generate using ModelOutputs ( #6735 )
...
* fix generate for GPT2 Double Head
* fix gpt2 double head model
* fix bart / t5
* also add for no beam search
* fix no beam search
* fix encoder decoder
* simplify t5
* simplify t5
* fix t5 tests
* fix BART
* fix transfo-xl
* fix conflict
* integrating sylvains and sams comments
* fix tf past_decoder_key_values
* fix enc dec test
2020-09-01 12:38:25 +02:00
Sam Shleifer
0ebc9699fa
[fixdoc] Add import to pegasus usage doc ( #6698 )
2020-08-24 15:54:57 -04:00
Stas Bekman
912a21ec78
remove BartForConditionalGeneration.generate ( #6659 )
...
As suggested here: https://github.com/huggingface/transformers/issues/6651#issuecomment-678594233
this removes generic `generate` doc with examples not-relevant to bart.
2020-08-25 00:42:34 +08:00
Suraj Patil
d0e42a7bed
CamembertForCausalLM ( #6577 )
...
* added CamembertForCausalLM
* add in __init__ and auto model
* style
* doc
2020-08-21 13:52:54 +02:00
Suraj Patil
fb6844aff5
[Pegasus Doc] minor typo ( #6579 )
...
Minor typo correction
@sshleifer
2020-08-18 12:47:47 -04:00
Sam Shleifer
12d7624199
[marian] converter supports models from new Tatoeba project ( #6342 )
2020-08-17 23:55:42 -04:00
Suraj Patil
c9564f5343
[Doc] add more MBart and other doc ( #6490 )
...
* add mbart example
* add Pegasus and MBart in readme
* typo
* add MBart in Pretrained models
* add pre-proc doc
* add DPR in readme
* fix indent
* doc fix
2020-08-17 12:30:26 -04:00
Patrick von Platen
36010cb1e2
fix pegasus doc ( #6533 )
2020-08-17 12:24:43 +02:00
Suraj Patil
680f1337c3
MBartForConditionalGeneration ( #6441 )
...
* add MBartForConditionalGeneration
* style
* rebase and fixes
* add mbart test in TEST_FILES_WITH_NO_COMMON_TESTS
* fix docs
* don't ignore mbart
* doc
* fix mbart fairseq link
* put mbart before bart
* apply doc suggestions
2020-08-14 03:21:16 -04:00
Patrick von Platen
0735def8e1
[EncoderDecoder] Add encoder-decoder for roberta/ vanilla longformer ( #6411 )
...
* add encoder-decoder for roberta
* fix headmask
* apply Sylvains suggestions
* fix typo
* Apply suggestions from code review
2020-08-12 18:23:30 +02:00
Sylvain Gugger
a8db954cda
Activate check on the CI ( #6427 )
...
* Activate check on the CI
* Fix repo inconsistencies
* Don't document too much
2020-08-12 08:42:14 -04:00
Sylvain Gugger
34fabe1697
Move prediction_loss_only to TrainingArguments ( #6426 )
2020-08-12 08:03:45 -04:00
Sam Shleifer
be1520d3a3
rename prepare_translation_batch -> prepare_seq2seq_batch ( #6103 )
2020-08-11 15:57:07 -04:00
Sam Shleifer
66fa8ceaea
PegasusForConditionalGeneration (torch version) ( #6340 )
...
Co-authored-by: Jingqing Zhang <jingqing.zhang15@imperial.ac.uk>
2020-08-11 14:31:23 -04:00
Patrick von Platen
00bb0b25ed
TF Longformer ( #5764 )
...
* improve names and tests longformer
* more and better tests for longformer
* add first tf test
* finalize tf basic op functions
* fix merge
* tf shape test passes
* narrow down discrepancies
* make longformer local attn tf work
* correct tf longformer
* add first global attn function
* add more global longformer func
* advance tf longformer
* finish global attn
* upload big model
* finish all tests
* correct false any statement
* fix common tests
* make all tests pass except keras save load
* fix some tests
* fix torch test import
* finish tests
* fix test
* fix torch tf tests
* add docs
* finish docs
* Update src/transformers/modeling_longformer.py
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
* Update src/transformers/modeling_tf_longformer.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* apply Lysandres suggestions
* reverse to assert statement because function will fail otherwise
* applying sylvains recommendations
* Update src/transformers/modeling_longformer.py
Co-authored-by: Sam Shleifer <sshleifer@gmail.com>
* Update src/transformers/modeling_tf_longformer.py
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Sam Shleifer <sshleifer@gmail.com>
2020-08-10 23:25:06 +02:00
Sylvain Gugger
6ba540b747
Add a script to check all models are tested and documented ( #6298 )
...
* Add a script to check all models are tested and documented
* Apply suggestions from code review
Co-authored-by: Kevin Canwen Xu <canwenxu@126.com>
* Address comments
Co-authored-by: Kevin Canwen Xu <canwenxu@126.com>
2020-08-07 09:18:37 -04:00
Sylvain Gugger
c67d1a0259
Tf model outputs ( #6247 )
...
* TF outputs and test on BERT
* Albert to DistilBert
* All remaining TF models except T5
* Documentation
* One file forgotten
* TF outputs and test on BERT
* Albert to DistilBert
* All remaining TF models except T5
* Documentation
* One file forgotten
* Add new models and fix issues
* Quality improvements
* Add T5
* A bit of cleanup
* Fix for slow tests
* Style
2020-08-05 11:34:39 -04:00
Kevin Canwen Xu
3c289fb38c
Remove outdated BERT tips ( #6217 )
...
* Remove out-dated BERT tips
* Update modeling_outputs.py
* Update bert.rst
* Update bert.rst
2020-08-04 01:17:56 +08:00
Faiaz Rahman
a39dfe4fb1
Fixed typo in Longformer ( #6180 )
2020-08-01 18:20:48 +08:00
Oren Amsalem
d24ea708d7
Actually the extra_id are from 0-99 and not from 1-100 ( #5967 )
...
a = tokenizer.encode("we got a <extra_id_99>", return_tensors='pt',add_special_tokens=True)
print(a)
>tensor([[ 62, 530, 3, 9, 32000]])
a = tokenizer.encode("we got a <extra_id_100>", return_tensors='pt',add_special_tokens=True)
print(a)
>tensor([[ 62, 530, 3, 9, 3, 2, 25666, 834, 23, 26,
834, 2915, 3155]])
2020-07-30 06:13:29 -04:00
Sylvain Gugger
a20969170b
Add AlbertForPretraining to doc ( #5914 )
2020-07-20 17:53:21 -04:00
Stas Bekman
45addfe96d
FlaubertForTokenClassification ( #5644 )
...
* implement FlaubertForTokenClassification as a subclass of XLMForTokenClassification
* fix mapping order
* add the doc
* add common tests
2020-07-13 14:59:53 -04:00
Sylvain Gugger
7fad617dc1
Document model outputs ( #5673 )
...
* Document model outputs
* Update docs/source/main_classes/output.rst
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
2020-07-10 17:31:02 -04:00
Sam Shleifer
353b8f1e7a
Add mbart-large-cc25, support translation finetuning ( #5129 )
...
improve unittests for finetuning, especially w.r.t testing frozen parameters
fix freeze_embeds for T5
add streamlit setup.cfg
2020-07-07 13:23:01 -04:00
Quentin Lhoest
fbd8792195
Add DPR model ( #5279 )
...
* beginning of dpr modeling
* wip
* implement forward
* remove biencoder + better init weights
* export dpr model to embed model for nlp lib
* add new api
* remove old code
* make style
* fix dumb typo
* don't load bert weights
* docs
* docs
* style
* move the `k` parameter
* fix init_weights
* add pretrained configs
* minor
* update config names
* style
* better config
* style
* clean code based on PR comments
* change Dpr to DPR
* fix config
* switch encoder config to a dict
* style
* inheritance -> composition
* add messages in assert startements
* add dpr reader tokenizer
* one tokenizer per model
* fix base_model_prefix
* fix imports
* typo
* add convert script
* docs
* change tokenizers conf names
* style
* change tokenizers conf names
* minor
* minor
* fix wrong names
* minor
* remove unused convert functions
* rename convert script
* use return_tensors in tokenizers
* remove n_questions dim
* move generate logic to tokenizer
* style
* add docs
* docs
* quality
* docs
* add tests
* style
* add tokenization tests
* DPR full tests
* Stay true to the attention mask building
* update docs
* missing param in bert input docs
* docs
* style
Co-authored-by: Lysandre <lysandre.debut@reseau.eseo.fr>
2020-07-07 08:56:12 -04:00
Patrick von Platen
d16e36c7e5
[Reformer] Add Masked LM Reformer ( #5426 )
...
* fix conflicts
* fix
* happy rebasing
2020-07-01 22:43:18 +02:00
Patrick von Platen
fe81f7d12c
finish reformer qa head ( #5433 )
2020-07-01 12:27:14 -04:00
Sylvain Gugger
1262495a91
Add TF auto model to the docs + fix sphinx warnings ( #5187 )
2020-06-22 14:43:52 -04:00
Vasily Shamporov
9a3f91088c
Add MobileBert ( #4901 )
...
* Add MobileBert
* Quality + Conversion script
* style
* Update src/transformers/modeling_mobilebert.py
* Links to S3
* Style
* TFMobileBert
Slight fixes to the pytorch MobileBert
Style
* MobileBertForMaskedLM (PT + TF)
* MobileBertForNextSentencePrediction (PT + TF)
* MobileFor{MultipleChoice, TokenClassification} (PT + TF)
ss
* Tests + Auto
* Doc
* Tests
* Addressing @sgugger's comments
* Adressing @patrickvonplaten's comments
* Style
* Style
* Integration test
* style
* Model card
Co-authored-by: Lysandre <lysandre.debut@reseau.eseo.fr>
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
2020-06-19 16:38:36 -04:00
Suraj Patil
18177a1a60
lm_labels => labels ( #5080 )
2020-06-18 09:16:29 +02:00
Sylvain Gugger
204ebc25e6
Update installation page and add contributing to the doc ( #5084 )
...
* Update installation page and add contributing to the doc
* Remove mention of symlinks
2020-06-17 14:01:10 -04:00
Sylvain Gugger
011cc0be51
Fix all sphynx warnings ( #5068 )
2020-06-16 16:50:02 -04:00
Yacine Jernite
49c5202522
Eli5 examples ( #4968 )
...
* add eli5 examples
* add dense query script
* query_di
* merging
* merging
* add_utils
* adds nearest neighbor wikipedia
* batch queries
* training_retriever
* new notebooks
* moved retriever traiing script
* finished wiki40b
* max_len_fix
* train_s2s
* retriever_batch_checkpointing
* cleanup
* merge
* dim_fix
* fix_indexer
* fix_wiki40b_snippets
* fix_embed_for_r
* fp32 index
* fix_sparse_q
* joint_training
* remove obsolete datasets
* add_passage_nn_results
* add_passage_nn_results
* add_batch_nn
* add_batch_nn
* add_data_scripts
* notebook
* notebook
* notebook
* fix_multi_gpu
* add_app
* full_caching
* full_caching
* notebook
* sparse_done
* images
* notebook
* add_image_gif
* with_Gif
* add_contr_image
* notebook
* notebook
* notebook
* train_functions
* notebook
* min_retrieval_length
* pandas_option
* notebook
* min_retrieval_length
* notebook
* notebook
* eval_Retriever
* notebook
* images
* notebook
* add_example
* add_example
* notebook
* fireworks
* notebook
* notebook
* joe's notebook comments
* app_update
* notebook
* notebook_link
* captions
* notebook
* assing RetriBert model
* add RetriBert to Auto
* change AutoLMHead to AutoSeq2Seq
* notebook downloads from hf models
* style_black
* style_black
* app_update
* app_update
* fix_app_update
* style
* style
* isort
* Delete WikiELI5training.ipynb
* Delete evaluate_eli5.py
* Delete WikiELI5explore.ipynb
* Delete ExploreWikiELI5Support.html
* Delete explainlikeimfive.py
* Delete wiki_snippets.py
* children before parent
* children before parent
* style_black
* style_black_only
* isort
* isort_new
* Update src/transformers/modeling_retribert.py
Co-authored-by: Julien Chaumond <chaumond@gmail.com>
* typo fixes
* app_without_asset
* cleanup
* Delete ELI5animation.gif
* Delete ELI5contrastive.svg
* Delete ELI5wiki_index.svg
* Delete choco_bis.svg
* Delete fireworks.gif
* Delete huggingface_logo.jpg
* Delete huggingface_logo.svg
* Delete Long_Form_Question_Answering_with_ELI5_and_Wikipedia.ipynb
* Delete eli5_app.py
* Delete eli5_utils.py
* readme
* Update README.md
* unused imports
* moved_info
* default_beam
* ftuned model
* disclaimer
* Update src/transformers/modeling_retribert.py
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
* black
* add_doc
* names
* isort_Examples
* isort_Examples
* Add doc to index
Co-authored-by: Julien Chaumond <chaumond@gmail.com>
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
Co-authored-by: Lysandre <lysandre.debut@reseau.eseo.fr>
2020-06-16 16:36:58 -04:00
Sylvain Gugger
f9f8a5312e
Add DistilBertForMultipleChoice ( #5032 )
...
* Add `DistilBertForMultipleChoice`
2020-06-15 18:31:41 -04:00
Suraj Patil
e93ccb3290
BartForQuestionAnswering ( #4908 )
2020-06-12 15:47:57 -04:00
Sylvain Gugger
538531cde5
Add AlbertForMultipleChoice ( #4959 )
...
* Add AlbertForMultipleChoice
* Make up to date and add all models to common tests
2020-06-12 14:20:19 -04:00
Suraj Patil
ef2dcdccaa
ElectraForQuestionAnswering ( #4913 )
...
* ElectraForQuestionAnswering
* udate __init__
* add test for electra qa model
* add ElectraForQuestionAnswering in auto models
* add ElectraForQuestionAnswering in all_model_classes
* fix outputs, input_ids defaults to None
* add ElectraForQuestionAnswering in docs
* remove commented line
2020-06-10 15:17:52 -04:00
Sylvain Gugger
41a1d27cde
Add XLMRobertaForQuestionAnswering ( #4855 )
...
* Add XLMRobertaForQuestionAnswering
* Formatting
* Make test happy
2020-06-08 21:22:37 -04:00
Sylvain Gugger
37be3786cf
Clean documentation ( #4849 )
...
* Clean documentation
2020-06-08 11:28:19 -04:00
Julien Chaumond
b42586ea56
Fix CI after killing archive maps ( #4724 )
...
* 🐛 Fix model ids for BART and Flaubert
2020-06-02 10:21:09 -04:00
Patrick von Platen
56ee2560be
[Longformer] Better handling of global attention mask vs local attention mask ( #4672 )
...
* better api
* improve automatic setting of global attention mask
* fix longformer bug
* fix global attention mask in test
* fix global attn mask flatten
* fix slow tests
* update docstring
* update docs and make more robust
* improve attention mask
2020-05-29 17:58:42 +02:00
Patrick von Platen
9c17256447
[Longformer] Multiple choice for longformer ( #4645 )
...
* add multiple choice for longformer
* add models to docs
* adapt docstring
* add test to longformer
* add longformer for mc in init and modeling auto
* fix tests
2020-05-29 13:46:08 +02:00
Patrick von Platen
c589eae2b8
[Longformer For Question Answering] Conversion script, doc, small fixes ( #4593 )
...
* add new longformer for question answering model
* add new config as well
* fix links
* fix links part 2
2020-05-26 14:58:47 +02:00
Patrick von Platen
3e3e552125
[Reformer] fix reformer num buckets ( #4564 )
...
* fix reformer num buckets
* fix
* adapt docs
* set num buckets in config
2020-05-25 16:04:45 -04:00
Alexander Measure
95a26fcf2d
link to paper was broken ( #4526 )
...
changed from https://https://arxiv.org/abs/2001.04451.pdf to https://arxiv.org/abs/2001.04451.pdf
2020-05-22 15:17:09 -04:00
Patrick von Platen
48c3a70b4e
[Longformer] Docs and clean API ( #4464 )
...
* add longformer docs
* improve docs
2020-05-19 21:52:36 +02:00
Soham Chatterjee
fa6113f9a0
Fixed spelling of training ( #4416 )
2020-05-18 11:23:29 -04:00
Sam Shleifer
9a687ebb77
[Marian Fixes] prevent predicting pad_token_id before softmax, support language codes, name multilingual models ( #4290 )
2020-05-13 17:29:41 -04:00
Sam Shleifer
3487be75ef
[Marian] documentation and AutoModel support ( #4152 )
...
- MarianSentencepieceTokenizer - > MarianTokenizer
- Start using unk token.
- add docs page
- add better generation params to MarianConfig
- more conversion utilities
2020-05-10 13:54:57 -04:00
Julien Chaumond
c99fe0386b
[doc] Fix broken links + remove crazy big notebook
2020-05-07 18:44:18 -04:00
