Yichao 'Peak' Ji
7a2e13204f
Remove duplicate declarations in dummy inputs for TFLongformer ( #21352 )
...
Remove duplicate declarations
2023-01-30 10:03:19 -05:00
简律纯
96addecff8
Corrected ( #21350 )
2023-01-30 09:38:15 -05:00
Wang, Yi
f3a7befffa
fix the issue that the output dict of jit model could not get [0] ( #21354 )
2023-01-30 09:23:55 -05:00
Yih-Dar
c749bd405e
Pipeline testing - using tiny models on Hub ( #20426 )
...
* rework pipeline tests
* run pipeline tests
* fix
* fix
* fix
* revert the changes in get_test_pipeline() parameter list
* fix expected error message
* skip a test
* clean up
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-01-30 10:39:43 +01:00
Yih-Dar
a582cfce3c
Fix GitModelIntegrationTest.test_batched_generation device issue ( #21362 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-01-30 10:37:56 +01:00
Maria Khalusova
73a2ff6974
Automated compatible models list for task guides ( #21338 )
...
* initial commit. added tip placeholders and a script
* removed unused imports, fixed paths
* fixed generated links
* make style
* split language modeling doc into two: causal language modeling and masked language modeling
* added check_task_guides.py to make fix-copies
* review feedback addressed
2023-01-27 13:19:28 -05:00
Lucain
8f3b4a1d5b
Little cleanup: let huggingface_hub manage token retrieval ( #21333 )
...
* Let huggingface_hub manage token retrieval
* flake8
* code quality
* adapt in every PushToHubMixin children
* add explicit return type
2023-01-27 12:09:49 -05:00
Arthur
0dff407d71
[Whisper] another patch ( #21324 )
...
* another patch
* fix timestamp test modeling
* let it be negative when the token is None
2023-01-27 16:35:16 +01:00
Yih-Dar
e5eb3e22ea
Fix RobertaPreLayerNorm doctest ( #21337 )
...
* add mask="<mask>"
* update
* update
* fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-01-27 16:20:25 +01:00
dependabot[bot]
36b668fa06
Bump onnx from 1.11.0 to 1.13.0 in /examples/research_projects/decision_transformer ( #21331 )
...
Bump onnx in /examples/research_projects/decision_transformer
Bumps [onnx](https://github.com/onnx/onnx ) from 1.11.0 to 1.13.0.
- [Release notes](https://github.com/onnx/onnx/releases )
- [Changelog](https://github.com/onnx/onnx/blob/main/docs/Changelog.md )
- [Commits](https://github.com/onnx/onnx/compare/v1.11.0...v1.13.0 )
---
updated-dependencies:
- dependency-name: onnx
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-27 10:13:13 -05:00
Michael Benayoun
938f437c53
Fix M2M100 positional embedding creation for ONNX ( #21328 )
...
* Fix M2M100 positional embedding creation for ONNX
* Restore READMEs
* Trigger CI
2023-01-27 10:43:19 +01:00
altryne
7d2a5fa749
Update Hebrew language code to he per IANA registry ( #21310 )
...
Here's my original PR into whisper that changes the same:
https://github.com/openai/whisper/pull/401
Per [IANA registry](https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry ), `iw` was deprecated as the code for Hebrew in 1989 and the preferred code is `he`
The correct subtag:
```
%%
Type: language
Subtag: he
Description: Hebrew
Added: 2005-10-16
Suppress-Script: Hebr
%%
```
And the deprecation
```
%%
Type: language
Subtag: iw
Description: Hebrew
Added: 2005-10-16
Deprecated: 1989-01-01
Preferred-Value: he
Suppress-Script: Hebr
%%
```
2023-01-26 13:34:39 -05:00
Younes Belkada
b225ee6ea0
[Doctest] Fix Perceiver doctest ( #21318 )
...
fix `Perceiver` doctest
2023-01-26 17:16:37 +01:00
Joao Gante
2b8feffad5
Generate: better compute_transition_scores examples ( #21323 )
2023-01-26 16:06:05 +00:00
Yih-Dar
449df41f01
Fix TFEncoderDecoder tests ( #21301 )
...
remove max_length=None
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-01-26 16:56:42 +01:00
Yih-Dar
857bad6e53
check paths in utils/documentation_tests.txt ( #21315 )
...
* check paths in utils/documentation_tests.txt
* check paths in utils/documentation_tests.txt
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-01-26 15:33:47 +01:00
Nicolas Patry
fd0ef8b66d
Small QoL for qa. ( #21316 )
2023-01-26 14:50:09 +01:00
Wonhyeong Seo
a01dd3818f
[i18n-KO] Translated quicktour page to Korean ( #20946 )
...
docs: ko: quicktour page
review by @ArthurZucker
docs: fix: remove duplicate
Co-Authored-By: Arthur <48595927+ArthurZucker@users.noreply.github.com>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
2023-01-26 14:10:02 +01:00
Yih-Dar
31336dcf3f
Fix 2 paths in the doctest list ( #21314 )
...
fix the list
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-01-26 12:07:08 +01:00
Yih-Dar
4e41b87e3d
Use model_class.__name__ and compare against XXX_MAPPING_NAMES ( #21304 )
...
* update
* update all
* clean up
* make quality
* clean up
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-01-26 11:31:31 +01:00
amyeroberts
d18a1cba24
Accept batched tensor of images as input to image processor ( #21144 )
...
* Accept a batched tensor of images as input
* Add to all image processors
* Update oneformer
2023-01-26 10:15:26 +00:00
Arthur
6f3faf3863
[WHISPER] Small patch ( #21307 )
...
* add small patch
* update tests, forced decoder ids is not prioritary against generation config
* fix two new tests
2023-01-25 22:49:23 +01:00
Nick Hill
140c6edeb9
Small fix to ExponentialDecayLengthPenalty docstring ( #21308 )
...
Currently, it incorrectly states that the exponential_decay_length_penalty tuple parameter is optional.
Also changed the corresponding type hint to be more specific.
2023-01-25 14:46:08 -05:00
Anahita Bhiwandiwalla
3a6e4a221c
Add BridgeTower model ( #20775 )
...
* Commit with BTModel and latest HF code
* Placeholder classes for BTForMLM and BTForITR
* Importing Bert classes from transformers
* Removed objectives.py and dist_utils.py
* Removed swin_transformer.py
* Add image normalization, BridgeTowerForImageAndTextRetrieval
* Add center_crop
* Removing bert tokenizer and LCI references
* Tested config loading from HF transformers hub
* Removed state_dict updates and added path to hub
* Enable center crop
* Getting image_size from config, renaming num_heads and num_layers
* Handling max_length in BridgeTowerProcessor
* Add BridgeTowerForMaskedLM
* Add doc string for BridgeTowerConfig
* Add doc strings for BT config, processor, image processor
* Adding docs, removed swin
* Removed convert_bridgetower_original_to_pytorch.py
* Added doc files for bridgetower, removed is_vision
* Add support attention_mask=None and BridgeTowerModelOutput
* Fix formatting
* Fixes with 'make style', 'make quality', 'make fixup'
* Remove downstream tasks from BridgeTowerModel
* Formatting fixes, add return_dict to BT models
* Clean up after doc_test
* Update BTModelOutput return type, fix todo in doc
* Remove loss_names from init
* implement tests and update tuples returned by models
* Add image reference to bridgetower.mdx
* after make fix-copies, make fixup, make style, make quality, make repo-consistency
* Rename class names with BridgeTower prefix
* Fix for image_size in BTImageProcessor
* implement feature extraction bridgetower tests
* Update image_mean and image_std to be list
* remove unused import
* Removed old comments
* Rework CLIP
* update config in tests followed config update
* Formatting fixes
* Add copied from for BridgeTowerPredictionHeadTransform
* Update bridgetower.mdx
* Update test_feature_extraction_bridgetower.py
* Update bridgetower.mdx
* BridgeTowerForMaskedLM is conditioned on image too
* Add BridgeTowerForMaskedLM
* Fixes
* Call post_init to init weights
* Move freeze layers into method
* Remove BTFeatureExtractor, add BT under multimodal models
* Remove BTFeatureExtractor, add BT under multimodal models
* Code review feedback - cleanup
* Rename variables
* Formatting and style to PR review feedback
* Move center crop after resize
* Use named parameters
* Style fix for modeling_bridgetower.py
* Update docs/source/en/model_doc/bridgetower.mdx
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update docs/source/en/model_doc/bridgetower.mdx
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update docs/source/en/model_doc/bridgetower.mdx
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/bridgetower/modeling_bridgetower.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/bridgetower/modeling_bridgetower.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update docs/source/en/model_doc/bridgetower.mdx
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
* Update src/transformers/models/bridgetower/modeling_bridgetower.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Rename config params, copy BERT classes, clean comments
* Cleanup irtr
* Replace Roberta imports, add BTTextConfig and Model
* Update docs, add visionconfig, consistent arg names
* make fixup
* Comments for forward in BTModel and make fixup
* correct tests
* Remove inconsistent roberta copied from
* Add BridgeTowerTextModel to dummy_pt_objects.py
* Add BridgeTowerTextModel to IGNORE_NON_TESTED
* Update docs for BT Text and Vision Configs
* Treat BridgeTowerTextModel as a private model
* BridgeTowerTextModel as private
* Run make fix-copies
* Adding BTTextModel to PRIVATE_MODELS
* Fix for issue with BT Text and Image configs
* make style changes
* Update README_ja.md
Add から to BridgeTower's description
* Clean up config, .mdx and arg names
* Fix init_weights. Remove nn.Sequential
* Formatting and style fixes
* Re-add tie_word_embeddings in config
* update test implementation
* update style
* remove commented out
* fix style
* Update README with abs for BridgeTower
* fix style
* fix mdx file
* Update bridgetower.mdx
* Update img src in bridgetower.mdx
* Update README.md
* Update README.md
* resolve style failed
* Update _toctree.yml
* Update README_ja.md
* Removed mlp_ratio, rename feats, rename BTCLIPModel
* Replace BTCLIP with BTVisionModel,pass in vision_config to BTVisionModel
* Add test_initialization support
* Add support for output_hidden_states
* Update support for output_hidden_states
* Add support for output_attentions
* Add docstring for output_hidden_states
* update tests
* add bridgetowervisionmodel as private model
* rerun the PR test
* Remove model_type, pass configs to classes, renames
* Change self.device to use weight device
* Remove image_size
* Style check fixes
* Add hidden_size and num_hidden_layers to BridgeTowerTransformer
* Update device setting
* cosmetic update
* trigger test again
* trigger tests again
* Update test_modeling_bridgetower.py
trigger tests again
* Update test_modeling_bridgetower.py
* minor update
* re-trigger tests
* Update docs/source/en/model_doc/bridgetower.mdx
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Remove pad, update max_text_len, doc cleanup, pass eps to LayerNorm
* Added copied to, some more review feedback
* make fixup
* Use BridgeTowerVisionEmbeddings
* Code cleanup
* Fixes for BridgeTowerVisionEmbeddings
* style checks
* re-tests
* fix embedding
* address comment on init file
* retrigger tests
* update import prepare_image_inputs
* update test_image_processing_bridgetower.py to reflect test_image_processing_common.py
* retrigger tests
Co-authored-by: Shaoyen Tseng <shao-yen.tseng@intel.com>
Co-authored-by: Tiep Le <tiep.le@intel.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
Co-authored-by: Tiep Le <97980157+tileintel@users.noreply.github.com>
2023-01-25 14:04:32 -05:00
Arthur
39799fbf85
[CI-Daily] replace past in prepare inputs for generation ( #21296 )
...
replace `past` in prepare inputs for generation
2023-01-25 18:25:59 +01:00
Maria Khalusova
238449414f
Documentation code sample fixes ( #21302 )
...
* Fixed the following:
pipe -> pipeline
out in pipe(data()) is a list of dict, not a dict
* Fixed the TypeError: __init__() missing 1 required positional argument: 'key'
* Added a tip: code sample requires additional libraries to run
* Fixed custom config's name
* added seqeval to the required libraries
* fixed a missing dependency,
fixed metric naming,
added checkpoint to fix the datacollator
* added checkpoint to fix the datacollator,
added missing dependency
2023-01-25 11:33:39 -05:00
Younes Belkada
015443f42b
[Doctest] Fix Blenderbot doctest ( #21297 )
...
fix blenderbot doctest
- add correct expected value
2023-01-25 17:28:29 +01:00
Yih-Dar
cc714d74c4
Update OneFormerModelIntegrationTest expected values ( #21295 )
...
* update values
* update values
* update values
* Update tests/models/oneformer/test_modeling_oneformer.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-01-25 17:27:02 +01:00
Younes Belkada
63b204eadd
[Hubert] Fix Hubert processing auto ( #21299 )
...
* fix Hubert processing auto
* remove unneeded space
2023-01-25 16:36:31 +01:00
Yih-Dar
de2d793e83
Fix EfficientFormer ( #21294 )
...
* fix
* fix checkpoint
* fix style
* tiny update
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-01-25 16:09:15 +01:00
Nicolas Patry
8788fd0ceb
Moving to cleaner tokenizer version or oneformer. ( #21292 )
...
Moving to cleaner tokenizer version.
2023-01-25 15:46:10 +01:00
Arthur
255257f3ea
[Whisper] Refactor whisper ( #21252 )
...
* update whisper logit processor
* add generate for whisper
* remove part of the whisper specific code from pipeline
* update logit processes
* major update
* enforce first timestamp
* update generate
* add more tests
* update new decoding strategy
* Apply suggestions from code review
* update docstring
* fixup
* default config will not have multilingual ar
* update expected tokenizer size, see pull on the hub for whisper-tiny
2023-01-25 13:09:43 +01:00
NielsRogge
f83135eb76
[Mask2Former] Add doc tests ( #21232 )
...
* Add doc tests
* Add OneFormer resourcesé
* Fix merge
* Fix style
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
2023-01-25 12:34:43 +01:00
Nicolas Patry
99e7905422
Supporting ImageProcessor in place of FeatureExtractor for pipelines ( #20851 )
...
* Fixing the pipeline with image processor.
* Update the slow test.
* Using only the first image processor.
* Include exclusion mecanism for Image processor.
* Do not handle Gitconfig, deemed as a bug.
* Apply suggestions from code review
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Remove `conversational` changes. They are not supposed to be here.
* Address first row of comments.
* Remove OneFormer modifications.
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
2023-01-25 10:16:31 +01:00
NielsRogge
efdbad56ab
[GIT] Add test for batched generation ( #21282 )
...
* Add test
* Apply suggestions
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
2023-01-25 10:14:18 +01:00
Steven Liu
de1ca3a0c5
Update expected values for doctest ( #21284 )
...
update expected values
2023-01-24 13:32:31 -08:00
Frederico Tommasi Caroli
1f981215dd
Fix TrainingArguments.label_names docs to reflect the correct default value behaviour ( #21288 )
...
* Update TrainingArguments.label_names docs
* Change wording
* Change wording
2023-01-24 14:48:24 -05:00
Sanchit Gandhi
14d058b940
[W2V2 with LM] Fix decoder test with params ( #21277 )
2023-01-24 19:27:56 +01:00
Arthur
94a7edd938
[GenerationConfig] add additional kwargs handling ( #21269 )
...
* add additional kwargs handling
* fix issue when serializing
* correct order of kwargs removal for serialization in from dict
* add `dict_torch_dtype_to_str` in case a dtype is needed for generation
* add condition when adding the kwargs : not from config
* Add comment based on review
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
* add test function
* default None when poping arg
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
2023-01-24 19:04:42 +01:00
Stas Bekman
9286039c2a
[examples/deepspeed] fix renamed api ( #21283 )
2023-01-24 09:54:33 -08:00
Younes Belkada
e2e393c6f2
[t5] Fix T5 inference in float16 + bnb error ( #21281 )
...
* attempts to fix:
- upcast input for `T5DenseActDense`
- add the condition `self.wo.weight.dtype != torch.int8`
- added tests on `test/mixed_int8`
- `make fixup`
* fix ci test
2023-01-24 18:14:38 +01:00
Alara Dirik
f424b09410
Fix MaskFormerImageProcessor.post_process_instance_segmentation ( #21256 )
...
* fix instance segmentation post processing
* add Mask2FormerImageProcessor
2023-01-24 18:49:29 +03:00
Hirokazu Kiyomaru
767939af52
Use logger.info instead of print to emit a logging message in hub.py ( #21273 )
...
use logger.info() instead of print() to emit a debug message
2023-01-24 10:37:10 -05:00
Nicolas Patry
67316444b0
Hotifx remove tuple for git config image processor. ( #21278 )
2023-01-24 16:07:50 +01:00
Matt
071529bd54
Use return_tensors="np" instead of "tf" ( #21266 )
...
Return NP instead of TF tensors for our data loading pipeline
2023-01-24 13:37:49 +00:00
Younes Belkada
f0fc791298
[Doc] fix broken link ( #21276 )
...
fix broken link
2023-01-24 11:18:48 +01:00
Yih-Dar
bde7378bf0
Skip test_multi_gpu_data_parallel_forward for UperNetModelTest ( #21216 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-01-24 10:41:16 +01:00
Sylvain Gugger
7119bb052a
v4.27.0.dev0
2023-01-23 16:52:35 -05:00
Sylvain Gugger
fd5cdaeea6
Models docstring ( #21225 )
...
* Clean all models
* Style
* Last to remove
* address review comments
* Address review comments
2023-01-23 14:33:18 -05:00
Maria Khalusova
9e86c4e193
Supported pipeline tasks update ( #21268 )
...
* added tasks from SUPPORTED_TASKS to docstrings
* make style
* sorted the tasks in the docstrtings in alphabetical order
2023-01-23 14:23:20 -05:00
