Commit Graph

9203 Commits

Author SHA1 Message Date
Omar Sanseviero
802984ad42
Fix and document Zero Shot Image Classification (#16079) 2022-03-14 08:50:36 +01:00
lewtun
6e1e88fd38
Add TFCamembertForCausalLM and ONNX integration test (#16073)
* Make Camembert great again!

* Add Camembert to TensorFlow ONNX tests
2022-03-14 08:40:42 +01:00
Thomas Chaigneau
20ab1582cf
Add missing type hints for all flavors of LayoutLMv2 PyTorch models. (#16089)
* Add missing type hints for all flavors of LayoutLMv2 PyTorch models.

* Fixed return types and added type hints for LayoutLM.

* Fix removed arguments which breaks tests.
2022-03-13 18:54:01 +00:00
James Barry
65cf33e7e5
Add type hints to XLM model (PyTorch) (#16108) 2022-03-12 19:28:48 +00:00
João Gustavo A. Amorim
841620684b
apply unpack_input decorator to ViT model (#16102) 2022-03-12 15:05:13 +00:00
p-mishra1
62b05b6917
Add type annotations for segformer classes (#16099) 2022-03-12 12:37:09 +00:00
Abdelrhman-Hosny
9042dfe35c
add unpack_inputs decorator to mbart (#16097) 2022-03-12 12:30:43 +00:00
Omar Sanseviero
3e9d0f7f59
Change unpacking of TF Bart inputs (#16094) 2022-03-12 12:06:55 +00:00
Stas Bekman
580dd87c55
[Deepspeed] add support for bf16 mode (#14569)
* [WIP] add support for bf16 mode

* prep for bf16

* prep for bf16

* fix; zero2/bf16 is ok

* check bf16 is available

* test fixes

* enable zero3_bf16

* config files

* docs

* split stage_dtype; merge back to non-dtype-specific config file

* fix doc

* cleanup

* cleanup

* bfloat16 => bf16 to match the PR changes

* s/zero_gather_fp16_weights_on_model_save/zero_gather_16bit_weights_on_model_save/; s/save_fp16_model/save_16bit_model/

* test fixes/skipping

* move

* fix

* Update docs/source/main_classes/deepspeed.mdx

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* backticks

* cleanup

* cleanup

* cleanup

* new version

* add note about grad accum in bf16

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2022-03-11 17:53:53 -08:00
Jeff Rasley
c1f209dadd
[ZeRO] Fixes issue with embedding resize (#16093)
* gather z3 params for new_lm_head

* Update src/transformers/modeling_utils.py

Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>

Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
2022-03-11 15:13:11 -08:00
Steven Liu
ae2dd42be5
Audio/vision task guides (#15808)
* 📝 first draft of audio/vision guides

*  make fixup

* 🖍 fix typo

* 🖍 close parentheses

* 🖍 apply feedback

* 🖍 apply feedback, make fixup

* 🖍 more fixup for perceiver

* 🖍 apply feedback

*  make fixup

* 🖍 fix data collator
2022-03-11 16:43:49 -06:00
Yih-Dar
cb5e50c8c2
[Fix doc example] FSMT (#16085)
* fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-03-11 21:21:31 +01:00
Thomas Chaigneau
eaed6897da
Add missing type hints for all flavors of RoBERTa PyTorch models. (#16086)
* Add missing type hints for all flavors of RoBERTa PyTorch models.

* Fixed type hints for all classes and fixed return types.
2022-03-11 19:40:50 +00:00
Lysandre Debut
a01fe4cd32
Rebuild deepspeed (#16081)
* Rebuild deepspeed

* Apply suggestions from code review

Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>

Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
2022-03-11 14:35:48 -05:00
João Gustavo A. Amorim
7f3d4440d6
add type annotations for ImageGPT (#16088) 2022-03-11 19:16:14 +00:00
Steven Liu
5b4c97d09d
Update troubleshoot guide (#16001)
* 📝 first draft

* 🖍 apply feedback

* 🖍 apply feedback
2022-03-11 13:05:44 -06:00
Kevin Bondzio
9442b3ce31
Add soft length regulation for sequence generation (#15245)
* add possibility to softly regulate length when using sampling method in model.generate() function

* fix test config, fix formatting

* fix rag integration, fix docstyling

* fix wrong docstring

* change param to tuple, add test

* fix old param in rag_model, remove unused import

* change test according to new param

* fix formatting

* fix test case

* fix doc style

* move start_length calculation to Logitprocessor

* add possibility to softly regulate length when using sampling method in model.generate() function

* fix rag integration, fix docstyling

* fix test config, fix formatting

* change param to tuple, add test

* fix old param in rag_model, remove unused import

* add possibility to softly regulate length when using sampling method in model.generate() function

* change param to tuple, add test

* fix old param in rag_model, remove unused import

* remove unused import

* fix small errors

* fix test

* add possibility to softly regulate length when using sampling method in model.generate() function

* fix test config, fix formatting

* fix rag integration, fix docstyling

* change param to tuple, add test

* fix old param in rag_model, remove unused import

* change test according to new param

* fix test case

* move start_length calculation to Logitprocessor

* add possibility to softly regulate length when using sampling method in model.generate() function

* fix rag integration, fix docstyling

* fix test config, fix formatting

* change param to tuple, add test

* fix old param in rag_model, remove unused import

* add possibility to softly regulate length when using sampling method in model.generate() function

* fix test config, fix formatting

* fix rag integration, fix docstyling

* add possibility to softly regulate length when using sampling method in model.generate() function

* fix rag integration, fix docstyling

* change param to tuple, add test

* fix old param in rag_model, remove unused import

* fix small errors

* Update src/transformers/generation_utils.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/generation_utils.py

* Update src/transformers/generation_utils.py

* fix docstring, add type ind model rag

* fix docstrings

* introduce seq_length variable for cleaner code

* fix black formatting

* add input_ids_seq_length to modeling_rag

* add input_ids_seq_length to test

* retrigger checks

* retrigger checks

Co-authored-by: Kevin Bondzio <kev@AIM-LAP-02.local>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Kevin Bondzio <kev@AIM-LAP-02.fritz.box>
2022-03-11 19:36:44 +01:00
Patrick von Platen
322c8533d7
Run daily test without time-out at least once (#16077) 2022-03-11 18:04:17 +01:00
feifang24
7e00247fad
check for key 'torch.dtype' in nested dicts in config (#16065) 2022-03-11 12:00:11 -05:00
Matt
5d2fed2e8c
Adding type hints for TFRoBERTa (#16057)
* Adding type annotations for TFRoBERTa

* Add type hints to TFRobertaModel too
2022-03-11 16:13:47 +00:00
Matt
bb69d154c5
Add type annotations for BERT and copies (#16074)
* Add type annotations for BERT and copies

* make fixup
2022-03-11 16:13:29 +00:00
Sylvain Gugger
f7708e1bed Force default brnahc name via the config 2022-03-11 10:09:15 -05:00
Sylvain Gugger
ecf989ca73 Trigger doc build 2022-03-11 09:20:05 -05:00
Lysandre Debut
0868fdef85
Fix torch-scatter version (#16072) 2022-03-11 09:03:27 -05:00
Funtowicz Morgan
5b369dc5d8
Remove assertion over possible activation functions in DistilBERT (#16066)
* Remove assertion over possible activation functions

* Same for TF and Flax
2022-03-11 14:27:59 +01:00
Sylvain Gugger
f5741bcd02
Move QDQBert in just PyTorch block (#16062) 2022-03-11 07:58:02 -05:00
Yih-Dar
b6bdb943b2
Fix a TF test name (LayoutLMModelTest) (#16061)
* fix name

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-03-11 11:22:36 +01:00
David S. Batista
96ac7549cb
updating fine-tune classifier documentation (#16063) 2022-03-10 16:21:56 -05:00
lewtun
6b09328368
Fix duplicate arguments passed to dummy inputs in ONNX export (#16045)
* Fix duplicate arguments passed to dummy inputs in ONNX export

* Fix M2M100 ONNX config

* Ensure we check PreTrained model only if torch is available

* Remove TensorFlow tests for models without PyTorch parity
2022-03-10 20:19:45 +01:00
Suraj Patil
ba21001f4c
support new marian models (#15831)
* support not sharing embeddings

* update modeling

* update tokenizer

* fix conversion script

* always use self.shared

* boom boom

* begin tests

* update tests

* fix resize_decoder_token_embeddings

* address Patrick's comments

* style

* update conversion script

* fix conversion script

* fix tokenizer

* better name target vocab

* add integration test for tokenizer with two vocabs

* style

* address Patrick's comments

* add integration test for model
2022-03-10 19:41:56 +01:00
Lysandre Debut
e66743e6c9
DeBERTa/DeBERTa-v2/SEW Support for torch 1.11 (#16043)
* Support for torch 1.11

* Address Sylvain's comment
2022-03-10 09:01:05 -05:00
Sanchit Gandhi
741e49305d
Fix Bug in Flax Seq2Seq Models (#16021)
* Fix Bug in Flax Seq2Seq Models

* incorporate suggested changes
2022-03-10 14:58:05 +01:00
Joao Gante
b7018abf3c
TF: Unpack model inputs through a decorator (#15907)
* MVP

* apply decorator to TFBertModel

* finish updating bert

* update rembert (copy-linked to bert)

* update roberta (copy-linked to bert); Fix args

* Now working for non-text modalities
2022-03-10 13:31:35 +00:00
Sylvain Gugger
19597998f6
Don't compute metrics in LM examples on TPU (#16029) 2022-03-10 07:44:51 -05:00
Sylvain Gugger
10591399d6
Build the doc in a seperate folder then move it (#16020)
* Build the doc in a seperate folder then move it

* Allow job

* Is this it?

* Dislike comments?

* Copy instead of move

* Removing version built

* Typos

* No variable

* Take _versions.yml into account

* Finish main job and add dev job

* Forgot the run

* Fix syntax error

* Execute builder from the repo

* Typo
2022-03-10 07:44:29 -05:00
Yih-Dar
2f463effb3
Fix TFDebertaV2ConvLayer in TFDebertaV2Model (#16031)
* fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-03-10 12:23:46 +01:00
Sanchit Gandhi
1da84ae02c
Fix Bug in Flax-Speech-Encoder-Decoder Test (#16041)
* Fix Bug in Flax-Speech-Encoder-Decoder Test

* change thresholds for CPU precision
2022-03-10 12:09:29 +01:00
Suraj Patil
b2a1c994cb
[README] fix url for Preprocessing tutorial (#16042) 2022-03-10 12:09:05 +01:00
NielsRogge
8d83ebdf18
[Tests] Add attentions_option to ModelTesterMixin (#15909)
* Add attentions_option to common tester

* Fix tests, apply suggestion

* Apply suggestion from code review

Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
2022-03-10 12:00:30 +01:00
Patrick von Platen
6ce11c2c0f
[Docs] Improve PyTorch, Flax generate API (#15988)
* Move generate docs

* up

* Update docs/source/_toctree.yml

* correct

* correct some stuff

* correct tests

* more fixes

* finish generate

* add to doc stest

* finish

* finalize

* add warning to generate method
2022-03-10 11:54:45 +01:00
André Storhaug
0951d31788
Fix dependency error message in ServeCommand (#16033)
"uvicorn" is misspelled as "unicorn".
2022-03-10 11:35:26 +01:00
NielsRogge
0835119bf3
Add Document Image Transformer (DiT) (#15984)
* Add conversion script

* Improve script

* Fix bug

* Add option to push to hub

* Add support for classification models

* Update model name

* Upload feature extractor files first

* Remove hash checking

* Fix config

* Add id2label

* Add import

* Fix id2label file name

* Fix expected shape

* Add model to README

* Improve docs

* Add integration test and fix CI

* Fix code style

* Add missing init

* Add model to SPECIAL_MODULE_TO_TEST_MAP

Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
2022-03-10 11:34:44 +01:00
Sanchit Gandhi
6c9010ef63
Update README.md 2022-03-10 10:20:37 +01:00
Sanchit Gandhi
fde901877a
Freeze Feature Encoder in FlaxSpeechEncoderDecoder (#15997)
* Freeze Feature Encoder in FlaxSpeechEncoderDecoder

* add backprop test
2022-03-10 09:59:19 +01:00
Pavel Belevich
65f9653ed0
Fix warning message in ElectraForCausalLM (#16023) 2022-03-09 17:27:15 -05:00
Suraj Patil
a69e185074
add doctests for bart like seq2seq models (#15987)
* boom boom

* enable doctest for few seq2seq models

* add seq2seq models in documentation_tests.txt

* fix docstring blenderbot

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* fix seq classif doc sample

* don't check loss for seq classif examples

* +IGNORE_OUTPUT => +IGNORE_RESULT

* fix _SEQ_CLASS_EXPECTED_OUTPUT_SHAPE

* fix some docs

* more fixes

* last fix (hopefully)

* fix big bird gen example

* fix mbart gen example

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2022-03-09 20:30:38 +01:00
Sanchit Gandhi
b256f3518d
Add FlaxBartForCausalLM (#15995)
* add causal lm

* add CausalLM tests

* Add FlaxBartForCausalLM

* Add EncoderDecoder model tests

* change docstring

* make repo-consistency

* suggested changes

* remove jax ops

* correction

* rename pre-trained decoder model
2022-03-09 19:53:01 +01:00
lewtun
50dd314d93
Add ONNX export for ViT (#15658)
* Add ONNX support for ViT

* Refactor to use generic preprocessor

* Add vision dep to tests

* Extend ONNX slow tests to ViT

* Add dummy image generator

* Use model_type to determine modality

* Add deprecation warnings for tokenizer argument

* Add warning when overwriting the preprocessor

* Add optional args to docstrings

* Add minimum PyTorch version to OnnxConfig

* Refactor OnnxConfig class variables from CONSTANT_NAME to snake_case

* Add reasonable value for default atol

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2022-03-09 17:36:59 +01:00
Yih-Dar
b7fa1e3dee
Use tiny models for get_pretrained_model in TFEncoderDecoderModelTest (#15989)
* Use tiny model for TFRembertEncoderDecoderModelTest.get_pretrained_model()

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-03-09 17:16:25 +01:00
Shotaro Ishihara
8feede229c
Fix broken code blocks in README.md (#15967)
at transformers/examples/pytorch/contrastive-image-text
2022-03-09 17:07:52 +01:00