transformers/tests
Pablo Montalvo 50290cf7a0
Uniformize model processors (#31368)
* add initial design for uniform processors + align model

* add uniform processors for altclip + chinese_clip

* add uniform processors for blip + blip2

* fix mutable default 👀

* add configuration test

* handle structured kwargs w defaults + add test

* protect torch-specific test

* fix style

* fix

* rebase

* update processor to generic kwargs + test

* fix style

* add sensible kwargs merge

* update test

* fix assertEqual

* move kwargs merging to processing common

* rework kwargs for type hinting

* just get Unpack from extensions

* run-slow[align]

* handle kwargs passed as nested dict

* add from_pretrained test for nested kwargs handling

* [run-slow]align

* update documentation + imports

* update audio inputs

* protect audio types, silly

* try removing imports

* make things simpler

* simplerer

* move out kwargs test to common mixin

* [run-slow]align

* skip tests for old processors

* [run-slow]align, clip

* !$#@!! protect imports, darn it

* [run-slow]align, clip

* [run-slow]align, clip

* update common processor testing

* add altclip

* add chinese_clip

* add pad_size

* [run-slow]align, clip, chinese_clip, altclip

* remove duplicated tests

* fix

* add blip, blip2, bridgetower

Added tests for bridgetower which override common. Also modified common
tests to force center cropping if existing

* fix

* update doc

* improve documentation for default values

* add model_max_length testing

This parameter depends on tokenizers received.

* Raise if kwargs are specified in two places

* fix

* removed copied from

* match defaults

* force padding

* fix tokenizer test

* clean defaults

* move tests to common

* add missing import

* fix

* adapt bridgetower tests to shortest edge

* uniformize donut processor + tests

* add wav2vec2

* extend common testing to audio processors

* add testing + bert version

* propagate common kwargs to different modalities

* BC order of arguments

* check py version

* revert kwargs merging

* add draft overlap test

* update

* fix blip2 and wav2vec due to updates

* fix copies

* ensure overlapping kwargs do not disappear

* replace .pop by .get to handle duplicated kwargs

* fix copies

* fix missing import

* add clearly wav2vec2_bert to uniformized models

* fix copies

* increase number of features

* fix style

* [run-slow] blip, blip2, bridgetower, donut, wav2vec2, wav2vec2_bert

* [run-slow] blip, blip_2, bridgetower, donut, wav2vec2, wav2vec2_bert

* fix concatenation

* [run-slow] blip, blip_2, bridgetower, donut, wav2vec2, wav2vec2_bert

* Update tests/test_processing_common.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* 🧹

* address comments

* clean up + tests

* [run-slow] instructblip, blip, blip_2, bridgetower, donut, wav2vec2, wav2vec2_bert

---------

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
2024-10-02 10:41:08 +02:00
..
agents Decorator for easier tool building (#33439) 2024-09-18 11:07:51 +02:00
benchmark [Test refactor 1/5] Per-folder tests reorganization (#15725) 2022-02-23 15:46:28 -05:00
bettertransformer Fixed malapropism error (#26660) 2023-10-09 11:04:57 +02:00
deepspeed Revert PR 32299, flag users when Zero-3 was missed (#32851) 2024-08-16 12:35:41 -04:00
extended [tests] skip tests for xpu (#33553) 2024-09-19 19:28:04 +01:00
fixtures Implementation of SuperPoint and AutoModelForKeypointDetection (#28966) 2024-03-19 14:43:02 +00:00
fsdp 🚨🚨🚨 Update min version of accelerate to 0.26.0 (#32627) 2024-08-20 11:42:36 +02:00
generation Add MLLama (#33703) 2024-09-25 19:56:25 +02:00
models Uniformize model processors (#31368) 2024-10-02 10:41:08 +02:00
optimization fix: Fixed the 1st argument name in classmethods (#31907) 2024-07-11 12:11:50 +01:00
peft_integration FIX [CI]: Fix failing tests for peft integration (#29330) 2024-02-29 03:56:16 +01:00
pipelines Make ASR pipeline compliant with Hub spec + add tests (#33769) 2024-10-01 18:15:04 +01:00
quantization Hqq serialization (#33141) 2024-09-30 14:47:18 +02:00
repo_utils Refactor CI: more explicit (#30674) 2024-08-30 18:17:25 +02:00
sagemaker Fixed log messages that are resulting in TypeError due to too many arguments (#32017) 2024-07-17 10:56:44 +01:00
tokenization Fix for slow the bug tokenizer adding spaces to single id decodes (#32564) 2024-09-18 12:32:02 +02:00
trainer Add AdEMAMix optimizer (#33682) 2024-09-25 18:07:21 +01:00
utils Fix passing str dtype to static cache (#33741) 2024-10-01 09:50:17 +02:00
__init__.py GPU text generation: mMoved the encoded_prompt to correct device 2020-01-06 15:11:12 +01:00
test_backbone_common.py Align backbone stage selection with out_indices & out_features (#27606) 2023-12-20 18:33:17 +00:00
test_configuration_common.py Refactor: Removed un-necessary object base class (#32230) 2024-07-26 10:33:02 +02:00
test_feature_extraction_common.py Split common test from core tests (#24284) 2023-06-15 07:30:24 -04:00
test_image_processing_common.py Update kwargs validation for preprocess with decorator (#32024) 2024-08-06 11:33:05 +01:00
test_image_transforms.py fix: center_crop occasionally outputs off-by-one dimension matrix (#30934) 2024-05-21 13:56:52 +01:00
test_modeling_common.py Add MLLama (#33703) 2024-09-25 19:56:25 +02:00
test_modeling_flax_common.py add sdpa to ViT [follow up of #29325] (#30555) 2024-05-16 10:56:11 +01:00
test_modeling_tf_common.py Port IDEFICS to tensorflow (#26870) 2024-05-13 15:59:46 +01:00
test_pipeline_mixin.py Make ASR pipeline compliant with Hub spec + add tests (#33769) 2024-10-01 18:15:04 +01:00
test_processing_common.py Uniformize model processors (#31368) 2024-10-02 10:41:08 +02:00
test_sequence_feature_extraction_common.py Fix typo (#25966) 2023-09-05 10:12:25 +02:00
test_tokenization_common.py Enable padding_side as call time kwargs (#33385) 2024-09-13 11:58:38 +01:00