riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-31 02:02:21 +06:00
Code Issues Actions 33 Packages Projects Releases Wiki Activity
851a4f7088
transformers/tests/models
History
Leo Tronchon 851a4f7088
Idefics: Fix information leak with cross attention gate in modeling (#26839) 
* fix image_attention gate in idefics modeling

* update comment

* cleaner gating

* fix gate condition

* create attention gate once

* update comment

* update doc of cross-attention forward

* improve comment

* bring back no_images

* pass cross_attention_gate similarly  to no_images gate

* add information on gate shape

* fix no_images placement

* make tests for gate

* take off no_images logic

* update test based on comments

* raise value error if cross_attention_gate is None

* send cross_attention_gate to device

* Revert "send cross_attention_gate to device"

This reverts commit 054f842284.

* send cross_attention_gate to device

* fix device in test + nit

* fill hidden_states with zeros instead of multiplying with the gate

* style

* Update src/transformers/models/idefics/modeling_idefics.py

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* Update src/transformers/models/idefics/modeling_idefics.py

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

---------

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
2023-11-21 13:26:01 +01:00
..
albert [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
align [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
altclip [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
audio_spectrogram_transformer Add numpy alternative to FE using torchaudio (#26339) 2023-11-08 07:39:37 +00:00
auto Remove-auth-token (#27060) 2023-11-13 14:20:54 +01:00
autoformer [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
bark [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
bart device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
barthez [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
bartpho Update quality tooling for formatting (#21480) 2023-02-06 18:10:56 -05:00
beit [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
bert device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
bert_generation [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
bert_japanese [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
bertweet Update quality tooling for formatting (#21480) 2023-02-06 18:10:56 -05:00
big_bird [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
bigbird_pegasus device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
biogpt Copied from for test files (#26713) 2023-10-11 14:12:09 +02:00
bit Update old existing feature extractor references (#24552) 2023-06-29 10:17:36 +01:00
blenderbot device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
blenderbot_small device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
blip device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
blip_2 device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
bloom [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
bridgetower Refactor image processor testers (#25450) 2023-08-11 11:30:18 +01:00
bros Update tiny model information and pipeline tests (#26285) 2023-09-25 18:08:12 +02:00
byt5 [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
camembert [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
canine [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
chinese_clip [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
clap [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
clip [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
clipseg [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
clvp Add CLVP (#24745) 2023-11-10 13:49:10 +00:00
code_llama [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
codegen Device agnostic testing (#25870) 2023-10-24 16:49:26 +02:00
conditional_detr Refactor image processor testers (#25450) 2023-08-11 11:30:18 +01:00
convbert device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
convnext Refactor image processor testers (#25450) 2023-08-11 11:30:18 +01:00
convnextv2 Add TensorFlow implementation of ConvNeXTv2 (#25558) 2023-11-01 15:09:55 +00:00
cpm Fix PipelineTests skip conditions (#22320) 2023-03-22 20:02:24 +01:00
cpmant CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
ctrl device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
cvt Make more test models smaller (#25005) 2023-07-24 10:08:47 -04:00
data2vec Fix typo (#25966) 2023-09-05 10:12:25 +02:00
deberta CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
deberta_v2 [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
decision_transformer 🔥Rework pipeline testing by removing PipelineTestCaseMeta 🚀 (#21516) 2023-02-28 19:40:57 +01:00
deformable_detr device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
deit device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
deta Refactor image processor testers (#25450) 2023-08-11 11:30:18 +01:00
detr Fix Detr CI (#25972) 2023-09-05 11:19:56 +02:00
dinat Update old existing feature extractor references (#24552) 2023-06-29 10:17:36 +01:00
dinov2 [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
distilbert use pytest.mark directly (#27390) 2023-11-09 13:32:54 +01:00
dit Update old existing feature extractor references (#24552) 2023-06-29 10:17:36 +01:00
donut Refactor image processor testers (#25450) 2023-08-11 11:30:18 +01:00
dpr CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
dpt Add DINOv2 depth estimation (#26092) 2023-11-13 16:20:42 +00:00
efficientformer Refactor image processor testers (#25450) 2023-08-11 11:30:18 +01:00
efficientnet 🚨🚨🚨 Remove softmax for EfficientNetForImageClassification 🚨🚨🚨 (#25501) 2023-08-14 17:08:47 +01:00
electra CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
encodec Add # Copied from statements to audio feature extractors that use the floats_list function (#26581) 2023-10-04 17:09:48 +02:00
encoder_decoder [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
ernie device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
ernie_m [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
esm [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
falcon Refactor: Use Llama RoPE implementation for Falcon (#26933) 2023-11-03 11:05:55 +00:00
flaubert device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
flava [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
fnet [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
focalnet Update tiny models and pipeline tests (#23446) 2023-05-18 17:29:04 +02:00
fsmt device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
funnel Big TF test cleanup (#24282) 2023-06-16 15:40:49 +01:00
fuyu Skip some fuyu tests (#27553) 2023-11-17 10:35:04 +01:00
git [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
glpn Input data format (#25464) 2023-08-16 17:45:02 +01:00
gpt_bigcode CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
gpt_neo [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
gpt_neox CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
gpt_neox_japanese [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
gpt_sw3 [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
gpt2 [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
gptj [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
gptsan_japanese [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
graphormer Make more test models smaller (#25005) 2023-07-24 10:08:47 -04:00
groupvit [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
herbert [Tokenizer] Fix slow and fast serialization (#26570) 2023-10-18 16:30:53 +02:00
hubert Fix failing tests on main due to torch 2.1 (#26607) 2023-10-05 10:27:05 +02:00
ibert Update deprecated torch.range in test_modeling_ibert.py (#27355) 2023-11-08 20:58:36 +01:00
idefics Idefics: Fix information leak with cross attention gate in modeling (#26839) 2023-11-21 13:26:01 +01:00
imagegpt [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
informer [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
instructblip [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
jukebox device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
kosmos2 Fix Kosmos2Processor batch mode (#27323) 2023-11-06 19:05:50 +01:00
layoutlm [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
layoutlmv2 [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
layoutlmv3 [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
layoutxlm [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
led device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
levit Refactor image processor testers (#25450) 2023-08-11 11:30:18 +01:00
lilt [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
llama [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
longformer Copied from for test files (#26713) 2023-10-11 14:12:09 +02:00
longt5 CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
luke [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
lxmert [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
m2m_100 [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
marian [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
markuplm [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
mask2former [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
maskformer [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
mbart device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
mbart50 [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
mega device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
megatron_bert CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
megatron_gpt2 Move test model folders (#17034) 2022-05-03 14:42:02 +02:00
mgp_str [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
mistral [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
mluke 🚨🚨 🚨🚨 [Tokenizer] attemp to fix add_token issues🚨🚨 🚨🚨 (#23909) 2023-09-18 20:28:36 +02:00
mobilebert [CI-test_torch] skip test_tf_from_pt_safetensors for 4 models (#27481) 2023-11-14 10:34:03 +01:00
mobilenet_v1 Refactor image processor testers (#25450) 2023-08-11 11:30:18 +01:00
mobilenet_v2 Refactor image processor testers (#25450) 2023-08-11 11:30:18 +01:00
mobilevit Refactor image processor testers (#25450) 2023-08-11 11:30:18 +01:00
mobilevitv2 Make more test models smaller (#25005) 2023-07-24 10:08:47 -04:00
mpnet Fix from_pt flag when loading with safetensors (#27394) 2023-11-13 15:18:19 +01:00
mpt Fix test_modeling_mpt typo in model id (#25606) 2023-08-21 11:11:21 +02:00
mra [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
mt5 Better TF docstring types (#23477) 2023-05-24 13:52:52 +01:00
musicgen MusicGen Update (#27084) 2023-11-08 13:26:02 +00:00
mvp [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
nat Update old existing feature extractor references (#24552) 2023-06-29 10:17:36 +01:00
nezha CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
nllb [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
nllb_moe [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
nougat Add Nougat (#25942) 2023-09-26 07:06:04 +02:00
nystromformer CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
oneformer [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
openai CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
opt device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
owlv2 [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
owlvit [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
pegasus [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
pegasus_x device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
perceiver [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
persimmon device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
phi [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
phobert Update quality tooling for formatting (#21480) 2023-02-06 18:10:56 -05:00
pix2struct [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
plbart device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
poolformer Refactor image processor testers (#25450) 2023-08-11 11:30:18 +01:00
pop2piano Update tiny model information and pipeline tests (#26285) 2023-09-25 18:08:12 +02:00
prophetnet CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
pvt device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
qdqbert CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
rag Big TF test cleanup (#24282) 2023-06-16 15:40:49 +01:00
realm CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
reformer [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
regnet Update old existing feature extractor references (#24552) 2023-06-29 10:17:36 +01:00
rembert CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
resnet Update old existing feature extractor references (#24552) 2023-06-29 10:17:36 +01:00
roberta Copied from for test files (#26713) 2023-10-11 14:12:09 +02:00
roberta_prelayernorm [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
roc_bert Copied from for test files (#26713) 2023-10-11 14:12:09 +02:00
roformer [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
rwkv CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
sam [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
seamless_m4t [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
segformer Refactor image processor testers (#25450) 2023-08-11 11:30:18 +01:00
sew Fix typo (#25966) 2023-09-05 10:12:25 +02:00
sew_d Fix typo (#25966) 2023-09-05 10:12:25 +02:00
speech_encoder_decoder Update quality tooling for formatting (#21480) 2023-02-06 18:10:56 -05:00
speech_to_text [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
speech_to_text_2 [CI-test_torch] skip test_tf_from_pt_safetensors for 4 models (#27481) 2023-11-14 10:34:03 +01:00
speecht5 [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
splinter CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
squeezebert CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
swiftformer Fix last models for common tests that are too big. (#25058) 2023-07-25 07:56:04 -04:00
swin Update old existing feature extractor references (#24552) 2023-06-29 10:17:36 +01:00
swin2sr [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
swinv2 Update old existing feature extractor references (#24552) 2023-06-29 10:17:36 +01:00
switch_transformers [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
t5 [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
table_transformer Fix last models for common tests that are too big. (#25058) 2023-07-25 07:56:04 -04:00
tapas [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
time_series_transformer [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
timesformer CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
timm_backbone [AutoBackbone] Add test (#26094) 2023-09-18 23:47:54 +02:00
transfo_xl Fix idx2sym not loaded from pretrained vocab file in Transformer XL (#27589) 2023-11-20 07:56:18 +01:00
trocr CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
tvlt remove failing tests and clean FE files (#27414) 2023-11-09 18:35:42 +00:00
umt5 [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
unispeech [Wav2Vec2 and Co] Update init tests for PT 2.1 (#26494) 2023-10-03 10:52:34 +02:00
unispeech_sat [Wav2Vec2 and Co] Update init tests for PT 2.1 (#26494) 2023-10-03 10:52:34 +02:00
upernet Fix last models for common tests that are too big. (#25058) 2023-07-25 07:56:04 -04:00
videomae Input data format (#25464) 2023-08-16 17:45:02 +01:00
vilt [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
vision_encoder_decoder [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
vision_text_dual_encoder [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
visual_bert [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
vit device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
vit_hybrid fix vit hybrid test (#25543) 2023-08-16 17:02:57 +02:00
vit_mae CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
vit_msn CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
vitdet Fix CPU offload + disk offload tests (#27204) 2023-11-01 19:25:23 +01:00
vitmatte [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
vits [Tokenizers] Skip tests temporarily (#26574) 2023-10-03 19:43:42 +02:00
vivit Input data format (#25464) 2023-08-16 17:45:02 +01:00
wav2vec2 [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
wav2vec2_conformer device agnostic models testing (#27146) 2023-10-31 18:12:14 +01:00
wav2vec2_phoneme Move test model folders (#17034) 2022-05-03 14:42:02 +02:00
wav2vec2_with_lm Fix some tests using "common_voice" (#27147) 2023-10-30 15:27:15 +01:00
wavlm [Wav2Vec2 and Co] Update init tests for PT 2.1 (#26494) 2023-10-03 10:52:34 +02:00
whisper [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
x_clip [core/ GC / tests] Stronger GC tests (#27124) 2023-10-30 19:53:46 +01:00
xglm [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
xlm CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
xlm_prophetnet [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
xlm_roberta [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
xlm_roberta_xl CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
xlnet [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00
xmod CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
yolos Refactor image processor testers (#25450) 2023-08-11 11:30:18 +01:00
yoso CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 2023-08-02 20:22:36 +02:00
__init__.py Move test model folders (#17034) 2022-05-03 14:42:02 +02:00