riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-31 02:02:21 +06:00
Code Issues Actions 23 Packages Projects Releases Wiki Activity
bb300ac686
transformers/tests/models
History
Arthur bb300ac686
Whisper Timestamp processor and prediction (#20620) 
* add draft logit processor

* add template functions

* update timesapmt processor parameters

* draft script

* simplify code

* cleanup

* fixup and clean

* update pipeline

* style

* clean up previous idea

* add tokenization utils

* update tokenizer and asr output

* fit whisper type

* style and update test

* clean test

* style test

* update tests

* update error test

* udpate code (not based on review yet)

* update tokenization

* update asr pipeline

* update code

* cleanup and update test

* fmt

* remove text verificatino

* cleanup

* cleanup

* add model test

* update tests

* update code add docstring

* update code and add docstring

* fix pipeline tests

* add draft logit processor

add template functions

update timesapmt processor parameters

draft script

simplify code

cleanup

fixup and clean

update pipeline

style

clean up previous idea

add tokenization utils

update tokenizer and asr output

fit whisper type

style and update test

clean test

style test

update tests

update error test

udpate code (not based on review yet)

update tokenization

update asr pipeline

update code

cleanup and update test

fmt

remove text verificatino

cleanup

cleanup

add model test

update tests

update code add docstring

update code and add docstring

fix pipeline tests

* Small update.

* Fixup.

* Tmp.

* More support.

* Making `forced_decoder_ids` non mandatory for users to set.

* update and fix first bug

* properly process sequence right after merge if last

* tofo

* allow list inputs + compute begin index better

* start adding tests

* add the 3 edge cases

* style

* format sequences

* fixup

* update

* update

* style

* test passes, edge cases should be good

* update last value

* remove Trie

* update tests and expec ted values

* handle bigger chunk_length

* clean tests a bit

* refactor chunk iter and clean pipeline

* update tests

* style

* refactor chunk iter and clean pipeline

* upade

* resolve comments

* Apply suggestions from code review

Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com>

* take stride right into account

* update test expected values

* Update code based on review

Co-authored-by: sgugger <sylvain.gugger@gmail.com>

Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com>
Co-authored-by: sgugger <sylvain.gugger@gmail.com>
2023-01-17 15:50:09 +01:00
..
albert Improve model tester (#19984) 2022-11-02 17:38:44 +01:00
altclip Fix torchscript tests for AltCLIP (#21102) 2023-01-13 10:03:19 +01:00
audio_spectrogram_transformer Add Audio Spectogram Transformer (#19981) 2022-11-21 18:58:54 +01:00
auto Make sure dynamic objects can be saved and reloaded (#21008) 2023-01-05 07:30:25 -05:00
bart Enable decoder_attention_mask in generate function (#20726) 2023-01-03 09:59:08 -05:00
barthez Move test model folders (#17034) 2022-05-03 14:42:02 +02:00
bartpho Move test model folders (#17034) 2022-05-03 14:42:02 +02:00
beit Rename test_feature_extraction files (#21140) 2023-01-17 14:04:07 +00:00
bert Fix past CI (#20967) 2023-01-12 18:04:21 +01:00
bert_generation Generate: move generation_*.py src files into generation/*.py (#20096) 2022-11-09 15:34:08 +00:00
bert_japanese update pyknp to rhoknp (#20890) 2022-12-31 01:22:26 -05:00
bertweet Move test model folders (#17034) 2022-05-03 14:42:02 +02:00
big_bird wrap forward passes with torch.no_grad() (#19273) 2022-10-04 16:13:22 +02:00
bigbird_pegasus Generate: move generation_*.py src files into generation/*.py (#20096) 2022-11-09 15:34:08 +00:00
biogpt Add BioGPT (#20420) 2022-12-05 10:12:03 -05:00
bit [Backbones] Improve out features (#20675) 2022-12-09 09:14:52 +01:00
blenderbot Generate: move generation_*.py src files into generation/*.py (#20096) 2022-11-09 15:34:08 +00:00
blenderbot_small Generate: move generation_*.py src files into generation/*.py (#20096) 2022-11-09 15:34:08 +00:00
blip [BLIP] Fix daily CI failing test (#20877) 2023-01-05 13:24:31 +01:00
bloom [Past CI] 🔥 Leave Past CI failures in the past 🔥 (#20861) 2022-12-27 18:37:25 +01:00
bort Move test model folders (#17034) 2022-05-03 14:42:02 +02:00
byt5 Replace as_target context managers by direct calls (#18325) 2022-07-29 08:09:09 -04:00
camembert Move test model folders (#17034) 2022-05-03 14:42:02 +02:00
canine Improve model tester (#19984) 2022-11-02 17:38:44 +01:00
chinese_clip Rename test_feature_extraction files (#21140) 2023-01-17 14:04:07 +00:00
clip Rename test_feature_extraction files (#21140) 2023-01-17 14:04:07 +00:00
clipseg [CLIPSeg] Fix integration test (#20995) 2023-01-05 14:30:32 +01:00
codegen Generate: move generation_*.py src files into generation/*.py (#20096) 2022-11-09 15:34:08 +00:00
conditional_detr Rename test_feature_extraction files (#21140) 2023-01-17 14:04:07 +00:00
convbert wrap forward passes with torch.no_grad() (#19274) 2022-10-04 16:12:03 +02:00
convnext Rename test_feature_extraction files (#21140) 2023-01-17 14:04:07 +00:00
cpm Move test model folders (#17034) 2022-05-03 14:42:02 +02:00
ctrl Generate: move generation_*.py src files into generation/*.py (#20096) 2022-11-09 15:34:08 +00:00
cvt [CvT] Tensorflow implementation (#18597) 2022-10-11 18:16:52 +01:00
data2vec update relative positional embedding (#20203) 2022-11-15 10:46:34 +01:00
deberta fix train_new_from_iterator in the case of byte-level tokenizers (#17549) 2022-06-08 15:30:41 +02:00
deberta_v2 🚨 🚨 🚨 Fix Issue 15003: SentencePiece Tokenizers Not Adding Special Tokens in convert_tokens_to_string (#15775) 2022-11-02 15:45:38 -04:00
decision_transformer Generate: move generation_*.py src files into generation/*.py (#20096) 2022-11-09 15:34:08 +00:00
deformable_detr Rename test_feature_extraction files (#21140) 2023-01-17 14:04:07 +00:00
deit Rename test_feature_extraction files (#21140) 2023-01-17 14:04:07 +00:00
detr Rename test_feature_extraction files (#21140) 2023-01-17 14:04:07 +00:00
dinat [NAT, DiNAT] Add backbone class (#20654) 2022-12-13 17:06:59 +01:00
distilbert Move test model folders (#17034) 2022-05-03 14:42:02 +02:00
dit Move test model folders (#17034) 2022-05-03 14:42:02 +02:00
donut Rename test_feature_extraction files (#21140) 2023-01-17 14:04:07 +00:00
dpr Replace as_target context managers by direct calls (#18325) 2022-07-29 08:09:09 -04:00
dpt Rename test_feature_extraction files (#21140) 2023-01-17 14:04:07 +00:00
electra Improve model tester (#19984) 2022-11-02 17:38:44 +01:00
encoder_decoder Generate: move generation_*.py src files into generation/*.py (#20096) 2022-11-09 15:34:08 +00:00
ernie update relative positional embedding (#20203) 2022-11-15 10:46:34 +01:00
esm Fix expected values for TF-ESM tests (#20680) 2022-12-08 15:26:09 +00:00
flaubert Improve model tester (#19984) 2022-11-02 17:38:44 +01:00
flava Rename test_feature_extraction files (#21140) 2023-01-17 14:04:07 +00:00
fnet wrap forward passes with torch.no_grad() (#19413) 2022-10-10 15:03:46 -04:00
fsmt Generate: move generation_*.py src files into generation/*.py (#20096) 2022-11-09 15:34:08 +00:00
funnel Update serving code to enable saved_model=True (#18153) 2022-07-22 18:05:38 +01:00
git [GIT] Fix training (#21133) 2023-01-16 15:37:38 +01:00
glpn Move test model folders (#17034) 2022-05-03 14:42:02 +02:00
gpt_neo Generate: move generation_*.py src files into generation/*.py (#20096) 2022-11-09 15:34:08 +00:00
gpt_neox skip some gpt_neox tests that require 80G RAM (#17923) 2022-07-01 09:04:38 -04:00
gpt_neox_japanese Add support for Japanese GPT-NeoX-based model by ABEJA, Inc. (#18814) 2022-09-14 10:17:40 -04:00
gpt_sw3 Add gpt-sw3 model to transformers (#20209) 2022-12-12 13:12:13 -05:00
gpt2 Fix past CI (#20967) 2023-01-12 18:04:21 +01:00
gptj Fix flax GPT-J-6B linking model in tests (#20556) 2022-12-05 14:00:05 +01:00
groupvit Allow passing arguments to model testers for CLIP-like models (#20044) 2022-11-04 18:01:41 +01:00
herbert Move test model folders (#17034) 2022-05-03 14:42:02 +02:00
hubert Update HubertModelIntegrationTest.test_inference_keyword_spotting (#20863) 2022-12-21 18:40:14 +01:00
ibert Improve model tester (#19984) 2022-11-02 17:38:44 +01:00
imagegpt Rename test_feature_extraction files (#21140) 2023-01-17 14:04:07 +00:00
jukebox Ci-jukebox (#20613) 2022-12-06 16:14:03 +01:00
layoutlm TF: TF 2.10 unpin + related onnx test skips (#18995) 2022-09-12 19:30:27 +01:00
layoutlmv2 Rename test_feature_extraction files (#21140) 2023-01-17 14:04:07 +00:00
layoutlmv3 Rename test_feature_extraction files (#21140) 2023-01-17 14:04:07 +00:00
layoutxlm Fix tapas scatter (#20149) 2022-11-14 01:04:26 -05:00
led Added missing test_tokenization_led (#20568) 2022-12-08 20:55:22 +01:00
levit Rename test_feature_extraction files (#21140) 2023-01-17 14:04:07 +00:00
lilt Generate: move generation_*.py src files into generation/*.py (#20096) 2022-11-09 15:34:08 +00:00
longformer Improve model tester (#19984) 2022-11-02 17:38:44 +01:00
longt5 Generate: move generation_*.py src files into generation/*.py (#20096) 2022-11-09 15:34:08 +00:00
luke Adding fine-tuning models to LUKE (#18353) 2022-08-01 11:09:47 -04:00
lxmert Update serving code to enable saved_model=True (#18153) 2022-07-22 18:05:38 +01:00
m2m_100 Generate: move generation_*.py src files into generation/*.py (#20096) 2022-11-09 15:34:08 +00:00
marian Generate: move generation_*.py src files into generation/*.py (#20096) 2022-11-09 15:34:08 +00:00
markuplm [processor] Add 'model input names' property (#20117) 2022-11-10 19:29:20 +00:00
mask2former Add Mask2Former (#20792) 2023-01-16 20:37:07 +03:00
maskformer Rename test_feature_extraction files (#21140) 2023-01-17 14:04:07 +00:00
mbart Enable PyTorch 1.13 (#20168) 2022-11-15 11:33:09 -05:00
mbart50 Replace as_target context managers by direct calls (#18325) 2022-07-29 08:09:09 -04:00
mctct [Past CI] 🔥 Leave Past CI failures in the past 🔥 (#20861) 2022-12-27 18:37:25 +01:00
megatron_bert Move test model folders (#17034) 2022-05-03 14:42:02 +02:00
megatron_gpt2 Move test model folders (#17034) 2022-05-03 14:42:02 +02:00
mluke Black preview (#17217) 2022-05-12 16:25:55 -04:00
mobilebert Fix train_step, test_step and tests for CLIP (#18684) 2022-09-09 20:01:02 +01:00
mobilenet_v1 Rename test_feature_extraction files (#21140) 2023-01-17 14:04:07 +00:00
mobilenet_v2 Rename test_feature_extraction files (#21140) 2023-01-17 14:04:07 +00:00
mobilevit Rename test_feature_extraction files (#21140) 2023-01-17 14:04:07 +00:00
mpnet Move test model folders (#17034) 2022-05-03 14:42:02 +02:00
mt5 [CI-Test] Fixes but also skips the mT5 tests (#20755) 2022-12-14 15:36:04 +01:00
mvp Generate: move generation_*.py src files into generation/*.py (#20096) 2022-11-09 15:34:08 +00:00
nat [NAT, DiNAT] Add backbone class (#20654) 2022-12-13 17:06:59 +01:00
nezha Generate: move generation_*.py src files into generation/*.py (#20096) 2022-11-09 15:34:08 +00:00
nllb Replace as_target context managers by direct calls (#18325) 2022-07-29 08:09:09 -04:00
nystromformer Move test model folders (#17034) 2022-05-03 14:42:02 +02:00
openai Generate: move generation_*.py src files into generation/*.py (#20096) 2022-11-09 15:34:08 +00:00
opt Generate: move generation_*.py src files into generation/*.py (#20096) 2022-11-09 15:34:08 +00:00
owlvit Rename test_feature_extraction files (#21140) 2023-01-17 14:04:07 +00:00
pegasus Generate: move generation_*.py src files into generation/*.py (#20096) 2022-11-09 15:34:08 +00:00
pegasus_x Generate: move generation_*.py src files into generation/*.py (#20096) 2022-11-09 15:34:08 +00:00
perceiver Update defaults and logic to match old FE (#20065) 2022-11-04 19:14:56 +00:00
phobert Move test model folders (#17034) 2022-05-03 14:42:02 +02:00
plbart Generate: fix plbart generation tests (#20391) 2022-11-22 17:56:04 +00:00
poolformer Rename test_feature_extraction files (#21140) 2023-01-17 14:04:07 +00:00
prophetnet Generate: move generation_*.py src files into generation/*.py (#20096) 2022-11-09 15:34:08 +00:00
qdqbert Move test model folders (#17034) 2022-05-03 14:42:02 +02:00
rag Avoid GPU OOM for a TF Rag test (#17638) 2022-06-10 18:50:29 +02:00
realm Fix RealmModelIntegrationTest.test_inference_open_qa (#21136) 2023-01-16 15:09:52 +01:00
reformer Generate: move generation_*.py src files into generation/*.py (#20096) 2022-11-09 15:34:08 +00:00
regnet [Tests] Improve test_attention_outputs (#20701) 2022-12-14 14:41:40 +01:00
rembert Move test model folders (#17034) 2022-05-03 14:42:02 +02:00
resnet [Tests] Improve test_attention_outputs (#20701) 2022-12-14 14:41:40 +01:00
retribert fix retribert's test_torch_encode_plus_sent_to_model (#17231) 2022-05-17 14:33:13 +02:00
roberta update relative positional embedding (#20203) 2022-11-15 10:46:34 +01:00
roberta_prelayernorm [RobertaPreLayernom] Fixes the CI daily test (#20886) 2022-12-23 19:55:17 +01:00
roc_bert fix both failing RoCBert tests (#20469) 2022-11-28 17:08:57 +01:00
roformer wrap forward passes with torch.no_grad() (#19438) 2022-10-10 14:54:54 -04:00
segformer Rename test_feature_extraction files (#21140) 2023-01-17 14:04:07 +00:00
sew Move test model folders (#17034) 2022-05-03 14:42:02 +02:00
sew_d Move test model folders (#17034) 2022-05-03 14:42:02 +02:00
speech_encoder_decoder Generate: move generation_*.py src files into generation/*.py (#20096) 2022-11-09 15:34:08 +00:00
speech_to_text [processor] Add 'model input names' property (#20117) 2022-11-10 19:29:20 +00:00
speech_to_text_2 Generate: move generation_*.py src files into generation/*.py (#20096) 2022-11-09 15:34:08 +00:00
splinter Fix Splinter test (#17854) 2022-06-24 16:26:14 +02:00
squeezebert Move test model folders (#17034) 2022-05-03 14:42:02 +02:00
swin [Past CI] 🔥 Leave Past CI failures in the past 🔥 (#20861) 2022-12-27 18:37:25 +01:00
swin2sr Add Swin2SR (#19784) 2022-12-16 16:24:01 +01:00
swinv2 Add swin transformer v2 (#17469) 2022-07-27 11:14:47 -04:00
switch_transformers [Switch Transformers] Fix failing slow test (#20346) 2022-11-21 15:36:49 +01:00
t5 Add keep_in_fp32_modules support (#20683) 2022-12-13 11:59:57 +01:00
table_transformer Generate: move generation_*.py src files into generation/*.py (#20096) 2022-11-09 15:34:08 +00:00
tapas Fix tapas scatter (#20149) 2022-11-14 01:04:26 -05:00
tapex Replace as_target context managers by direct calls (#18325) 2022-07-29 08:09:09 -04:00
time_series_transformer Add a decorator for flaky tests (#19498) 2022-10-12 14:00:17 -04:00
timesformer [New Model] Add TimeSformer model (#18908) 2022-12-02 09:13:25 +01:00
trajectory_transformer [Past CI] 🔥 Leave Past CI failures in the past 🔥 (#20861) 2022-12-27 18:37:25 +01:00
transfo_xl Generate: move generation_*.py src files into generation/*.py (#20096) 2022-11-09 15:34:08 +00:00
trocr Generate: move generation_*.py src files into generation/*.py (#20096) 2022-11-09 15:34:08 +00:00
unispeech Move test model folders (#17034) 2022-05-03 14:42:02 +02:00
unispeech_sat Move test model folders (#17034) 2022-05-03 14:42:02 +02:00
upernet Add UperNet (#20648) 2023-01-16 09:39:13 +01:00
van [Tests] Improve test_attention_outputs (#20701) 2022-12-14 14:41:40 +01:00
videomae Rename test_feature_extraction files (#21140) 2023-01-17 14:04:07 +00:00
vilt Rename test_feature_extraction files (#21140) 2023-01-17 14:04:07 +00:00
vision_encoder_decoder Fix donut image processor (#20625) 2022-12-08 19:10:40 +00:00
vision_text_dual_encoder [processor] Add 'model input names' property (#20117) 2022-11-10 19:29:20 +00:00
visual_bert wrap forward passes with torch.no_grad() (#19439) 2022-10-10 14:54:36 -04:00
vit Rename test_feature_extraction files (#21140) 2023-01-17 14:04:07 +00:00
vit_hybrid [ViTHybrid] Fix accelerate slow tests (#20679) 2022-12-08 17:39:32 +01:00
vit_mae TF: tests for (de)serializable models with resized tokens (#19013) 2022-09-16 16:38:08 +01:00
vit_msn Some fixes regarding auto mappings and test class names (#19923) 2022-10-27 14:38:59 +02:00
wav2vec2 [Past CI] 🔥 Leave Past CI failures in the past 🔥 (#20861) 2022-12-27 18:37:25 +01:00
wav2vec2_conformer [Test] Fix W2V-Conformer integration test (#17303) 2022-05-17 18:20:36 +02:00
wav2vec2_phoneme Move test model folders (#17034) 2022-05-03 14:42:02 +02:00
wav2vec2_with_lm [processor] Add 'model input names' property (#20117) 2022-11-10 19:29:20 +00:00
wavlm Move test model folders (#17034) 2022-05-03 14:42:02 +02:00
whisper Whisper Timestamp processor and prediction (#20620) 2023-01-17 15:50:09 +01:00
x_clip Allow passing arguments to model testers for CLIP-like models (#20044) 2022-11-04 18:01:41 +01:00
xglm Generate: move generation_*.py src files into generation/*.py (#20096) 2022-11-09 15:34:08 +00:00
xlm Generate: move generation_*.py src files into generation/*.py (#20096) 2022-11-09 15:34:08 +00:00
xlm_prophetnet Black preview (#17217) 2022-05-12 16:25:55 -04:00
xlm_roberta Black preview (#17217) 2022-05-12 16:25:55 -04:00
xlm_roberta_xl update relative positional embedding (#20203) 2022-11-15 10:46:34 +01:00
xlnet Generate: move generation_*.py src files into generation/*.py (#20096) 2022-11-09 15:34:08 +00:00
yolos Rename test_feature_extraction files (#21140) 2023-01-17 14:04:07 +00:00
yoso fix train_new_from_iterator in the case of byte-level tokenizers (#17549) 2022-06-08 15:30:41 +02:00
__init__.py Move test model folders (#17034) 2022-05-03 14:42:02 +02:00