Commit Graph

9241 Commits

Author SHA1 Message Date
Matt
cd4c5c9060
TF XLA greedy generation (#15786)
* First attempt at TF XLA generation

* Fix comments

* Update XLA greedy generate with direct XLA calls

* Support attention mask, prepare_inputs_for_generation no longer hardcoded for greedy

* Handle position_ids correctly

* make xla generate work for non xla case

* force using xla generate

* refactor

* more fixes

* finish cleaning

* finish

* finish

* clean gpt2 tests

* add gpt2 tests

* correct more cases

* up

* finish

* finish

* more fixes

* flake 8 stuff

* final rag fix

* Update src/transformers/models/rag/modeling_tf_rag.py

* finish t5 as well

* finish

* Update src/transformers/generation_utils.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2022-03-15 14:19:20 +01:00
Yih-Dar
e5bc438cc8
[Fix doc example] Fix 2 PyTorch Vilt docstring examples (#16076)
* fix 2 pytorch vilt docstring examples

* add vilt to doctest list file

* remove device

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-03-15 13:35:02 +01:00
Markus Sagen
bcaf566038
[Fix doc example] Fix first example for the custom_datasets tutorial (#16087)
* Fix inconsistent example variable naming

- Example code for a sequence classification in Tensorflow had spelling mistakes and incorrect and inconsistent naming
- Changed variable naming to be consistent with the two other TF examples

* Fix incorrect incorrect training examples
2022-03-15 08:17:51 -04:00
Sylvain Gugger
8bfd2fb8f0
Use templates (#16142)
* Use tempaltes for all doc building jobs

* Add this branch to the doc build

* Switch to main branch
2022-03-15 08:07:56 -04:00
Daniel Espejel
daa4944759
Added spanish translation of quicktour.mdx (#16158)
* Added spanish translation of quicktour.mdx

* Suggestions applied in the revision of the translation

Co-authored-by: Omar U. Espejel <espejelomar@gmail.com>

Co-authored-by: Omar U. Espejel <espejelomar@gmail.com>
2022-03-15 08:07:35 -04:00
Ahmed Elnaggar
57713443de
Configurable Relative Position Max. Distance (#16155)
* Configurable Relative Position Max. Distance

* fix missing config

Co-authored-by: ahmed-elnaggar <ahmed.elnaggar@allianz.com>
2022-03-15 08:05:33 -04:00
marxav
cd1ffb40bf
typo "conaining" -> "containing" (#16132) 2022-03-15 07:08:53 -04:00
Patrick von Platen
5664d27622
Shift responsibilities a bit (#16154) 2022-03-15 11:07:17 +01:00
Pavel Belevich
5a386fb05c
Make transformers.utils.fx. _SUPPORTED_MODELS unique (#16015) 2022-03-15 10:15:03 +01:00
NielsRogge
a7aca42fc4
Improve Swin for VisionEncoderDecoder (#16070)
* Add Swin2Bart test

* Fix swin tests

Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
2022-03-15 09:59:48 +01:00
Francesco Saverio Zuppichini
0a057201a9
Visual Attention Network (VAN) (#16027)
* encoder works

* addded files

* norm in stage

* convertion script

* tests

* fix copies

* make fix-copies

* fixed __init__

* make fix-copies

* fix

* shapiro test needed

* make fix-copie

* minor changes

* make style + quality

* minor refactor conversion script

* rebase + tests

* removed unused variables

* updated doc

* toctree

* CI

* doc

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* resolved conversations

* make fixup

* config passed to modules

* config passed to modules

* Apply suggestions from code review

Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>

* conversations

* conversations

* copyrights

* normal test

* tests

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
2022-03-15 08:47:12 +01:00
Dan Tegzes
8f3ea7a1e1
Add type hints for GPTNeo PyTorch (#16127)
* Add type hints for SqueezeBert PyTorch

* Add type hints for GPTNeo PyTorch

* style fixes

* chenged List with Tuple
2022-03-14 20:26:12 +01:00
Francesco Saverio Zuppichini
e3008c679f
[WIP] Resnet (#15770)
* first commit

* ResNet model correctly implemented.

basic modeling + weights conversion is done

removed unused doc

mdx file

doc and conversion script

added feature_extractor to auto

test

minor changes + style + quality

doc

test

Delete process.yml

A left over from my attempt of running circleci locally

* minor changes

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* new test format

* minor changes from conversations

* minor changes from conversations

* make style + quality

* readded the tests

* test + README

* minor changes from conversations

* error in README

* make fix-copies

* removed regression for classification head

* make quality

* fixed loss control flow

* fixed loss control flow

* resolved conversations

* Apply suggestions from code review

Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>

* READMEs

* index.mdx

* minor changes

* updated tests and models

* unused import

* outputs

* Update docs/source/model_doc/resnet.mdx

Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>

* added embeddings_size

* Apply suggestions from code review

Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>

* conversation

* added push to hub

* test

* embedding_size

* make fix-copies

* resolved conversations

* CI

* changed organization

* minor changes

* CI

* minor changes

* conversations

* conversation

* doc

* tests

* removed unused docstring

* conversation

* removed unused outputs

* CI

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
2022-03-14 19:57:55 +01:00
Kamal Raj
6458236181
TF Electra - clearer model variable naming (#16143) 2022-03-14 18:10:07 +00:00
Joydeep Bhattacharjee
37793259bb
update albert with tf decorator (#16147) 2022-03-14 18:09:19 +00:00
Sylvain Gugger
e109edf16f
Use HF_ENDPOINT for custom endpoints (#16139) 2022-03-14 13:26:23 -04:00
Martin Pan
0dcdfe8630
Add type hints for FNet PyTorch (#16123) 2022-03-14 17:11:19 +00:00
Jacob Dineen
f86235ad1b
Add type annotations for CLIP (torch) (#16059) (#16106)
* clip typhinting #16059

* removed optional type annotations for dataclass in CLIPOutput

* type annotation fixes per Rocket - Clip Torch
2022-03-14 16:56:04 +00:00
Lysandre Debut
c1000e703b
Dcoker images runtime -> devel (#16141)
* Runtime -> Devel

* Torch before DeepSpeed
2022-03-14 12:37:20 -04:00
Kamal Raj
10cf1ffdbf
Added missing type hints - ELECTRA TF (#16104)
* Add missing type hints - ELECTRA TF

* bool -> Optional[bool]
2022-03-14 16:28:34 +00:00
Dan Tegzes
6db8693086
Add type hints for SqueezeBert PyTorch (#16126)
* Add type hints for SqueezeBert PyTorch

* fixed unused List err

* style fixes
2022-03-14 16:21:08 +00:00
Hyeonsoo Lee
5493c10ecb
Add type hints for PoolFormer in Pytorch (#16121) 2022-03-14 16:14:04 +00:00
Bhavika Tekwani
6c2f3ed74c
Add type hints for Luke in PyTorch (#16111)
* Add type hints for LukeModel

* Add type hints for entitypairclassification

* Remove blank space

Co-authored-by: bhavika <bhavika@debian-BULLSEYE-live-builder-AMD64>
2022-03-14 15:55:03 +00:00
Michael Benayoun
37a9fc49f2
Choose framework for ONNX export (#16018)
* Can choose framework for ONNX export

* Fix docstring
2022-03-14 16:47:29 +01:00
Pepijn Boers
3f8360a7b6
Add type hints for TFDistilBert (#16107)
* Add type hints for TFDistilBert

* Update src/transformers/models/distilbert/modeling_tf_distilbert.py

Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>
2022-03-14 15:39:59 +00:00
Bhavika Tekwani
97e32b7854
Improve model variable naming - CLIP [TF] (#16128)
* First pass

* Fixup

* Fix broken tests

* Make unpack_inputs the first decorator
2022-03-14 15:26:40 +00:00
Bhavika Tekwani
d02bd4f333
Better input variable naming for OpenAI (TF) (#16129)
* Replace input_processing

* move unpack_inputs
2022-03-14 15:25:45 +00:00
Yih-Dar
c8c8c114a3
[Fix doc example] Fix checkpoint name in docstring example in Speech2Text2 (#16083)
* Fix checkpoint name in docstring example

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2022-03-14 16:19:18 +01:00
Kamal Raj
72ae06b904
Added missing type hints - V1 and V2 (#16105) 2022-03-14 15:12:22 +00:00
Kamal Raj
1d43933fbc
Added missing type hints (#16103) 2022-03-14 14:53:57 +00:00
Yhary Arias
efd6e9a82a
Spanish translation of the file training.mdx (#16047)
* Spanish translation of the file training.mdx

* Settings - Spanish translation of the file training.mdx

* Latest changes to the Spanish translation of the training.mdx file

* Delete Hugging.mdx

* Last changes to the training fil Espanish version

* Latest modifications

* Latest changes, document ready for PR

* Nits

Co-authored-by: Yhary Arias <yharystefa@gmail.com>
Co-authored-by: Omar U. Espejel <espejelomar@gmail.com>
2022-03-14 10:12:38 -04:00
NielsRogge
9fd584e544
Add copied from statements and fix prefix (#16119)
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
2022-03-14 15:05:14 +01:00
Merve Noyan
f284aa320d
steps strategy fix for PushtoHubCallback (#16138) 2022-03-14 13:37:07 +00:00
Minh Chien Vu
e3645fd280
Change unpacking of TF mobilebert inputs to use decorator (#16110)
* Change unpacking of TF mobilebert inputs to use decorator

* Move unpack_inputs as the top decorator

* make fixup

Co-authored-by: ChienVM <chien_vm@detomo.co.jp>
2022-03-14 13:15:08 +00:00
Yih-Dar
5dbf36bd4e
Fix ProphetNetTokenizer (#16082)
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-03-14 09:02:41 -04:00
Yih-Dar
923c35b5c5
Make TF pt-tf equivalence test more aggressive (#15839)
* Make TF pt-tf equivalence test more aggressive

* Fix for TFConvNextModelTest and TFTransfoXLModelTest

* fix kwargs for outputs

* clean-up

* Add docstring for check_outputs()

* remove: need to rename encoder-decoder

* clean-up

* send PyTorch things to the correct device

* Add back the accidentally removed test case in test_pt_tf_model_equivalence()

* Fix: change to tuple before calling check_outputs()

* Fix: tfo could be a list

* use to_tuple()

* allow tfo only to be tuple or tensor

* allow tfo to be list or tuple for now + style change

* minor fix

* remove np.copy and update comments

* tfo -> tf_output, same for pt

* Add more detailed comment

* remove the incorrect comment

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-03-14 13:31:32 +01:00
tiedemann
9e9f6b8a45
Update convert_marian_to_pytorch.py (#16124)
Configuration `tied-embeddings-all` implies `tied-embeddings-src`
2022-03-14 12:15:38 +01:00
Sanchit Gandhi
2de99e6c43
Fix Loading of Flax(Speech)EncoderDecoderModel kwargs from PreTrained Encoder-Decoder Checkpoints (#16056)
* Fix Loading of Flax(Speech)EncoderDecoderModel kwargs from PreTrained Encoder-Decoder Checkpoints

* change wording
2022-03-14 10:12:29 +01:00
Omar Sanseviero
802984ad42
Fix and document Zero Shot Image Classification (#16079) 2022-03-14 08:50:36 +01:00
lewtun
6e1e88fd38
Add TFCamembertForCausalLM and ONNX integration test (#16073)
* Make Camembert great again!

* Add Camembert to TensorFlow ONNX tests
2022-03-14 08:40:42 +01:00
Thomas Chaigneau
20ab1582cf
Add missing type hints for all flavors of LayoutLMv2 PyTorch models. (#16089)
* Add missing type hints for all flavors of LayoutLMv2 PyTorch models.

* Fixed return types and added type hints for LayoutLM.

* Fix removed arguments which breaks tests.
2022-03-13 18:54:01 +00:00
James Barry
65cf33e7e5
Add type hints to XLM model (PyTorch) (#16108) 2022-03-12 19:28:48 +00:00
João Gustavo A. Amorim
841620684b
apply unpack_input decorator to ViT model (#16102) 2022-03-12 15:05:13 +00:00
p-mishra1
62b05b6917
Add type annotations for segformer classes (#16099) 2022-03-12 12:37:09 +00:00
Abdelrhman-Hosny
9042dfe35c
add unpack_inputs decorator to mbart (#16097) 2022-03-12 12:30:43 +00:00
Omar Sanseviero
3e9d0f7f59
Change unpacking of TF Bart inputs (#16094) 2022-03-12 12:06:55 +00:00
Stas Bekman
580dd87c55
[Deepspeed] add support for bf16 mode (#14569)
* [WIP] add support for bf16 mode

* prep for bf16

* prep for bf16

* fix; zero2/bf16 is ok

* check bf16 is available

* test fixes

* enable zero3_bf16

* config files

* docs

* split stage_dtype; merge back to non-dtype-specific config file

* fix doc

* cleanup

* cleanup

* bfloat16 => bf16 to match the PR changes

* s/zero_gather_fp16_weights_on_model_save/zero_gather_16bit_weights_on_model_save/; s/save_fp16_model/save_16bit_model/

* test fixes/skipping

* move

* fix

* Update docs/source/main_classes/deepspeed.mdx

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* backticks

* cleanup

* cleanup

* cleanup

* new version

* add note about grad accum in bf16

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2022-03-11 17:53:53 -08:00
Jeff Rasley
c1f209dadd
[ZeRO] Fixes issue with embedding resize (#16093)
* gather z3 params for new_lm_head

* Update src/transformers/modeling_utils.py

Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>

Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
2022-03-11 15:13:11 -08:00
Steven Liu
ae2dd42be5
Audio/vision task guides (#15808)
* 📝 first draft of audio/vision guides

*  make fixup

* 🖍 fix typo

* 🖍 close parentheses

* 🖍 apply feedback

* 🖍 apply feedback, make fixup

* 🖍 more fixup for perceiver

* 🖍 apply feedback

*  make fixup

* 🖍 fix data collator
2022-03-11 16:43:49 -06:00
Yih-Dar
cb5e50c8c2
[Fix doc example] FSMT (#16085)
* fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-03-11 21:21:31 +01:00