lewtun
6b09328368
Fix duplicate arguments passed to dummy inputs in ONNX export ( #16045 )
...
* Fix duplicate arguments passed to dummy inputs in ONNX export
* Fix M2M100 ONNX config
* Ensure we check PreTrained model only if torch is available
* Remove TensorFlow tests for models without PyTorch parity
2022-03-10 20:19:45 +01:00
Suraj Patil
ba21001f4c
support new marian models ( #15831 )
...
* support not sharing embeddings
* update modeling
* update tokenizer
* fix conversion script
* always use self.shared
* boom boom
* begin tests
* update tests
* fix resize_decoder_token_embeddings
* address Patrick's comments
* style
* update conversion script
* fix conversion script
* fix tokenizer
* better name target vocab
* add integration test for tokenizer with two vocabs
* style
* address Patrick's comments
* add integration test for model
2022-03-10 19:41:56 +01:00
Lysandre Debut
e66743e6c9
DeBERTa/DeBERTa-v2/SEW Support for torch 1.11 ( #16043 )
...
* Support for torch 1.11
* Address Sylvain's comment
2022-03-10 09:01:05 -05:00
Sanchit Gandhi
741e49305d
Fix Bug in Flax Seq2Seq Models ( #16021 )
...
* Fix Bug in Flax Seq2Seq Models
* incorporate suggested changes
2022-03-10 14:58:05 +01:00
Joao Gante
b7018abf3c
TF: Unpack model inputs through a decorator ( #15907 )
...
* MVP
* apply decorator to TFBertModel
* finish updating bert
* update rembert (copy-linked to bert)
* update roberta (copy-linked to bert); Fix args
* Now working for non-text modalities
2022-03-10 13:31:35 +00:00
Sylvain Gugger
19597998f6
Don't compute metrics in LM examples on TPU ( #16029 )
2022-03-10 07:44:51 -05:00
Sylvain Gugger
10591399d6
Build the doc in a seperate folder then move it ( #16020 )
...
* Build the doc in a seperate folder then move it
* Allow job
* Is this it?
* Dislike comments?
* Copy instead of move
* Removing version built
* Typos
* No variable
* Take _versions.yml into account
* Finish main job and add dev job
* Forgot the run
* Fix syntax error
* Execute builder from the repo
* Typo
2022-03-10 07:44:29 -05:00
Yih-Dar
2f463effb3
Fix TFDebertaV2ConvLayer in TFDebertaV2Model ( #16031 )
...
* fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-03-10 12:23:46 +01:00
Sanchit Gandhi
1da84ae02c
Fix Bug in Flax-Speech-Encoder-Decoder Test ( #16041 )
...
* Fix Bug in Flax-Speech-Encoder-Decoder Test
* change thresholds for CPU precision
2022-03-10 12:09:29 +01:00
Suraj Patil
b2a1c994cb
[README] fix url for Preprocessing tutorial ( #16042 )
2022-03-10 12:09:05 +01:00
NielsRogge
8d83ebdf18
[Tests] Add attentions_option to ModelTesterMixin ( #15909 )
...
* Add attentions_option to common tester
* Fix tests, apply suggestion
* Apply suggestion from code review
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
2022-03-10 12:00:30 +01:00
Patrick von Platen
6ce11c2c0f
[Docs] Improve PyTorch, Flax generate API ( #15988 )
...
* Move generate docs
* up
* Update docs/source/_toctree.yml
* correct
* correct some stuff
* correct tests
* more fixes
* finish generate
* add to doc stest
* finish
* finalize
* add warning to generate method
2022-03-10 11:54:45 +01:00
André Storhaug
0951d31788
Fix dependency error message in ServeCommand ( #16033 )
...
"uvicorn" is misspelled as "unicorn".
2022-03-10 11:35:26 +01:00
NielsRogge
0835119bf3
Add Document Image Transformer (DiT) ( #15984 )
...
* Add conversion script
* Improve script
* Fix bug
* Add option to push to hub
* Add support for classification models
* Update model name
* Upload feature extractor files first
* Remove hash checking
* Fix config
* Add id2label
* Add import
* Fix id2label file name
* Fix expected shape
* Add model to README
* Improve docs
* Add integration test and fix CI
* Fix code style
* Add missing init
* Add model to SPECIAL_MODULE_TO_TEST_MAP
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
2022-03-10 11:34:44 +01:00
Sanchit Gandhi
6c9010ef63
Update README.md
2022-03-10 10:20:37 +01:00
Sanchit Gandhi
fde901877a
Freeze Feature Encoder in FlaxSpeechEncoderDecoder ( #15997 )
...
* Freeze Feature Encoder in FlaxSpeechEncoderDecoder
* add backprop test
2022-03-10 09:59:19 +01:00
Pavel Belevich
65f9653ed0
Fix warning message in ElectraForCausalLM ( #16023 )
2022-03-09 17:27:15 -05:00
Suraj Patil
a69e185074
add doctests for bart like seq2seq models ( #15987 )
...
* boom boom
* enable doctest for few seq2seq models
* add seq2seq models in documentation_tests.txt
* fix docstring blenderbot
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* fix seq classif doc sample
* don't check loss for seq classif examples
* +IGNORE_OUTPUT => +IGNORE_RESULT
* fix _SEQ_CLASS_EXPECTED_OUTPUT_SHAPE
* fix some docs
* more fixes
* last fix (hopefully)
* fix big bird gen example
* fix mbart gen example
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2022-03-09 20:30:38 +01:00
Sanchit Gandhi
b256f3518d
Add FlaxBartForCausalLM ( #15995 )
...
* add causal lm
* add CausalLM tests
* Add FlaxBartForCausalLM
* Add EncoderDecoder model tests
* change docstring
* make repo-consistency
* suggested changes
* remove jax ops
* correction
* rename pre-trained decoder model
2022-03-09 19:53:01 +01:00
lewtun
50dd314d93
Add ONNX export for ViT ( #15658 )
...
* Add ONNX support for ViT
* Refactor to use generic preprocessor
* Add vision dep to tests
* Extend ONNX slow tests to ViT
* Add dummy image generator
* Use model_type to determine modality
* Add deprecation warnings for tokenizer argument
* Add warning when overwriting the preprocessor
* Add optional args to docstrings
* Add minimum PyTorch version to OnnxConfig
* Refactor OnnxConfig class variables from CONSTANT_NAME to snake_case
* Add reasonable value for default atol
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2022-03-09 17:36:59 +01:00
Yih-Dar
b7fa1e3dee
Use tiny models for get_pretrained_model in TFEncoderDecoderModelTest ( #15989 )
...
* Use tiny model for TFRembertEncoderDecoderModelTest.get_pretrained_model()
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-03-09 17:16:25 +01:00
Shotaro Ishihara
8feede229c
Fix broken code blocks in README.md ( #15967 )
...
at transformers/examples/pytorch/contrastive-image-text
2022-03-09 17:07:52 +01:00
Francesco Saverio Zuppichini
1e8f37992f
done ( #16012 )
2022-03-09 15:51:56 +01:00
Basile Van Hoorick
38bce1d4cf
Make pos optional to avoid crashing PerceiverModel operation ( #15972 )
...
Updates `PerceiverAudioPreprocessor` `forward()` implementation to match most other preprocessors / postprocessors
2022-03-09 15:48:52 +01:00
Sylvain Gugger
cec89e1a0e
Simplify release utils ( #15921 )
...
* Simplify release utils
* Quality
2022-03-09 08:47:58 -05:00
Lysandre Debut
e493a3a5e2
Fix github actions comment ( #16009 )
...
* Add issue number
* Dev
2022-03-09 08:39:03 -05:00
Joao Gante
e7f34ccd4f
Swag example: Update doc format ( #16014 )
2022-03-09 13:25:34 +00:00
Yih-Dar
3ea046995e
Removed an outdated check about hdf5_version ( #16011 )
...
* removed an outdated check about hdf5_version
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-03-09 14:21:23 +01:00
Patrick von Platen
c1aaa43935
[Doctests] Move doctests to new GPU & Fix bugs ( #15969 )
...
* test
* up
* up
* Empty test commit
* up
* update tests
* up
* fix some vision models
* correct
* correct docs
* Trigger notification
* finalize
* check
* correct quicktour
* Apply suggestions from code review
* improve doctests
* Trigger Build
* next try
* next try
* and again
* Output current clone information
* Output current clone information
* Correct path
* add tf round again
* revert to daily job
Co-authored-by: Lysandre <lysandre.debut@reseau.eseo.fr>
2022-03-09 13:09:56 +01:00
Nicolas Patry
f4e4ad34cc
Add ForInstanceSegmentation models to image-segmentation pipelines ( #15937 )
...
* Adding ForInstanceSegmentation to pipelines.
* Last fix `category_id` renamed to `label_id`.
* Can't be none no more.
* No `is_thing_map` anymore.
2022-03-09 10:19:05 +01:00
David Hall
5b7dcc7342
Seed _get_train_sampler's generator with arg seed to improve reproducibility ( #15961 )
...
* Seed get_train_sampler's generator with arg seed to improve reproducibility
and make the world_size<=1 code path more similar to the others
* move test file into trainer test explicitly
* dumb typo
* make style lint happy
* per discussion, switch to data_seed
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2022-03-08 13:45:41 -05:00
Joao Gante
70203b5937
TF generate refactor - past without encoder outputs ( #15944 )
...
* Remove packed past from generation_tf_utils
* update models with the new past format
* update template accordingly
2022-03-08 14:46:44 +00:00
Joao Gante
62d847602a
Update TF multiple choice example ( #15868 )
2022-03-08 13:16:34 +00:00
Patrick von Platen
ab2f8d12a7
add hf hub to env version command ( #15981 )
2022-03-08 14:03:03 +01:00
Yih-Dar
72983303c5
Fix TFEncoderDecoderModelTest - Pytorch device ( #15979 )
...
* fix device
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-03-08 13:37:20 +01:00
Sylvain Gugger
f5a080dd10
Do a pull in case docs were updated during build ( #15922 )
2022-03-08 07:19:41 -05:00
Yeb Havinga
91fb62d01c
Speedup training by using numpy instead of jnp for batch shuffling ( #15963 )
...
Speedup training by using numpy instead of jnp for batch shuffling
Co-authored-by: Yeb Havinga <y.t.havinga@mgrid.net>
2022-03-08 12:18:38 +01:00
Nicolas Patry
ea07064a5c
Returning outputs only when asked for for MaskFormer. ( #15936 )
...
* Returning outputs only when asked for for MaskFormer.
* Adding `output_auxiliary_logits` to the config.
2022-03-08 11:17:57 +01:00
NielsRogge
b19f3e69a0
[Tests] Fix ViTMAE integration test ( #15949 )
...
* Fix test across both cpu and gpu
* Fix typo
2022-03-08 10:49:44 +01:00
NielsRogge
9879a1d5f0
Fix LayoutLMv2 test ( #15939 )
...
* Fix LayoutLMv2 test
* Update black
2022-03-08 10:49:30 +01:00
Yih-Dar
8b9ae45549
Set scale_embedding to False in some TF tests ( #15952 )
...
* set scale_embedding to False to avoid large (> 1e-5) output differences between PT/TF
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-03-07 22:14:33 +01:00
Steven Liu
38cc35069c
Update training scripts docs ( #15931 )
...
* 📝 first draft
* 🖍 apply feedback
* 🖍 remove examples from toctree
* 🗑 remove examples from docs/source
2022-03-07 13:29:14 -06:00
Sylvain Gugger
c87cfd653c
Better error message when inputs are empty
2022-03-07 13:29:16 -05:00
Francesco Saverio Zuppichini
e9fa7cd5d7
Make is_thing_map in Feature Extractor post_process_panoptic_segmentation defaults to all instances ( #15954 )
...
* is_thing_map defaults to all instances
* better naming
* control flow
* resolving conversations
2022-03-07 19:10:32 +01:00
Sanchit Gandhi
2596f95e84
Fix Embedding Module Bug in Flax Models ( #15920 )
2022-03-07 18:17:45 +01:00
Sanchit Gandhi
1a62b25caf
Backprop Test for Freeze FlaxWav2Vec2 Feature Encoder ( #15938 )
...
* Backprop Test for Freeze FlaxWav2Vec2 Feature Encoder
* remove jnp.ndarray type suggestion
* assert frozen grads are precisely zero
2022-03-07 18:10:15 +01:00
Konstantin Dobler
544fd9876b
Support modern list type hints in HfArgumentParser ( #15951 )
...
* Support modern list type hint in HfArgumentParser
* Fix formatting with black
2022-03-07 10:22:48 -05:00
Suraj Patil
60b81dfa6f
remove re-defination of FlaxWav2Vec2ForCTCModule ( #15965 )
2022-03-07 14:58:44 +01:00
Chan Woo Kim
ef9c3ca348
[Bug Fix] Beam search example in docs fails & a fix (integrating max_length in BeamScorer.finalize()) ( #15555 )
...
* added the test and fix
* had left out a comment
2022-03-07 09:10:18 +01:00
Francesco Saverio Zuppichini
9932ee4b4b
made MaskFormerModelTest faster ( #15942 )
2022-03-04 19:11:48 +01:00
