transformers/utils
Raushan Turganbay 21d5025826
Attn implementation for composite models (#32238)
* first try

* codestyle

* idefics2 is happy

* [run-slow] llava, llava_next, video_llava, vipllava, llava_next_video, idefics, idefics2, kosmos2, fuyu, blip, blip_2, instructblip, instructblipvideo, paligemma

* fix-copies

* [run-slow] llava, llava_next, video_llava, vipllava, llava_next_video, idefics, idefics2, kosmos2, fuyu, blip, blip_2, instructblip, instructblipvideo

* blip-2 needs to init vision from config

* when was this removed O_o

* minor fix

* tests

* this way?

* tests

* model-agnostic code

* codestyle

* add tests for idefics

* modify general test for VLMs

* no generation test for vlm yet!

* no generation test here also

* wanr in VIT-SDPA if output attn

* add more tests

* user can pass dict as attn impl

* repo consistency

* update

* muicgen

* no prints

* forgot speech enc-dec and clip

* how many composite models we have?

* musicgen meelody is same as mudicgen

* +siglip

* fix tests + add some more

* remove idefics custom overriden code

* make idefics2 automappable

* nits

* skip tests

* doctests

* Update src/transformers/models/idefics2/configuration_idefics2.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* Update tests/models/clip/test_modeling_clip.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* Update tests/models/idefics2/test_modeling_idefics2.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* Update tests/models/idefics2/test_modeling_idefics2.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* Update src/transformers/configuration_utils.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* major update, no need for automap

* clean up

* add FA2 test

* more tests

* style

* skip tests

* why did these started failing now?

* no attributes for FA2 needed

* one tiny test

* address comment about FA2 false warning

* style

* add new models and resolve conflicts

* fix copies

* let it be this way for now, come back tomorrow to review

* some more fixes

* update

* more updates

* update

* fix copies

* style and tests

* another big update

* fix tests

* fix tests

* update

* another update

* fix tests

* fix copies

* fix tests

---------

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
2024-10-22 06:54:44 +02:00
..
test_module AutoImageProcessor (#20111) 2022-11-08 19:54:41 +00:00
tf_ops Check TF ops for ONNX compliance (#10025) 2021-02-15 07:55:10 -05:00
add_pipeline_model_mapping_to_test.py update ruff version (#30932) 2024-05-22 06:40:15 +02:00
check_bad_commit.py Ping team members for new failed tests in daily CI (#34171) 2024-10-17 16:11:52 +02:00
check_build.py Fix import of FalconMambaForCausalLM (#33381) 2024-09-10 09:14:54 +02:00
check_config_attributes.py Fix Gradient Accumulation issue (#34191) 2024-10-17 22:34:40 +02:00
check_config_docstrings.py Granitemoe (#33207) 2024-09-21 01:43:50 +02:00
check_copies.py Generate: visit non-llm prepare_inputs_for_generation (#34199) 2024-10-17 16:53:48 +01:00
check_doc_toc.py update ruff version (#30932) 2024-05-22 06:40:15 +02:00
check_docstrings.py Add Flax Dinov2 (#31960) 2024-08-19 09:28:13 +01:00
check_doctest_list.py update ruff version (#30932) 2024-05-22 06:40:15 +02:00
check_dummies.py update ruff version (#30932) 2024-05-22 06:40:15 +02:00
check_inits.py Fix import of FalconMambaForCausalLM (#33381) 2024-09-10 09:14:54 +02:00
check_model_tester.py Add a new script to check model testers' config (#22063) 2023-03-13 19:11:19 +01:00
check_modular_conversion.py Modular transformers: modularity and inheritance for new model additions (#33248) 2024-09-24 15:54:07 +02:00
check_repo.py Attn implementation for composite models (#32238) 2024-10-22 06:54:44 +02:00
check_self_hosted_runner.py Tiny fix for check_self_hosted_runner.py (#24052) 2023-06-06 18:17:41 +02:00
check_support_list.py [RoBERTa-based] Add support for sdpa (#30510) 2024-08-28 10:26:00 +02:00
check_table.py Add OmDet-Turbo (#31843) 2024-09-25 13:26:28 -04:00
check_tf_ops.py Check TF ops for ONNX compliance (#10025) 2021-02-15 07:55:10 -05:00
create_dependency_mapping.py Modular transformers: modularity and inheritance for new model additions (#33248) 2024-09-24 15:54:07 +02:00
create_dummy_models.py CI: fix efficientnet pipeline timeout and prevent future similar issues due to large image size (#33123) 2024-08-27 11:58:27 +01:00
custom_init_isort.py Import structure & first three model refactors (#31329) 2024-09-10 11:10:53 +02:00
deprecate_models.py Remove copied froms for deprecated models (#31153) 2024-06-03 09:42:53 +01:00
download_glue_data.py update ruff version (#30932) 2024-05-22 06:40:15 +02:00
extract_warnings.py update github actions packages' version to suppress warnings (#30249) 2024-04-15 15:08:09 +02:00
get_ci_error_statistics.py Add artifact name in job step to maintain job / artifact correspondence (#28682) 2024-01-31 15:58:17 +01:00
get_github_job_time.py Make Slack CI reporting stronger (#21823) 2023-02-28 17:12:44 +01:00
get_modified_files.py exclude deleted files in the fixup script (#21436) 2023-02-03 12:57:02 -05:00
get_previous_daily_ci.py Ping team members for new failed tests in daily CI (#34171) 2024-10-17 16:11:52 +02:00
get_test_info.py CI: fix efficientnet pipeline timeout and prevent future similar issues due to large image size (#33123) 2024-08-27 11:58:27 +01:00
important_models.txt ENH: [CI] Add new workflow to run slow tests of important models on push main if they are modified (#29235) 2024-04-12 10:01:28 +02:00
models_to_deprecate.py update ruff version (#30932) 2024-05-22 06:40:15 +02:00
modular_model_converter.py Add support for inheritance from class with different suffix in modular (#34077) 2024-10-15 14:55:09 +02:00
not_doctested.txt Add support for __all__ and potentilly deleting functions (#33859) 2024-10-08 10:19:17 +02:00
notification_service_doc_tests.py Refactor doctest (#30210) 2024-04-15 13:20:36 +02:00
notification_service_quantization.py Revive Nightly/Past CI (#31159) 2024-06-20 18:57:24 +02:00
notification_service.py Ping team members for new failed tests in daily CI (#34171) 2024-10-17 16:11:52 +02:00
past_ci_versions.py (Re-)Enable Nightly + Past CI (#22393) 2023-03-30 21:06:35 +02:00
patch_helper.py [Patch helper] update to not have to checkout main (#34006) 2024-10-09 09:21:46 +02:00
pr_slow_ci_models.py Avoid duplication in PR slow CI model list (#30634) 2024-05-03 18:19:30 +02:00
print_env.py Print more library versions in CI (#17384) 2022-06-02 10:24:16 +02:00
process_bad_commit_report.py Ping team members for new failed tests in daily CI (#34171) 2024-10-17 16:11:52 +02:00
process_test_artifacts.py fix the parallel number of CI nodes when it is smaller than number of tests (#33276) 2024-09-03 16:53:21 +02:00
release.py update ruff version (#30932) 2024-05-22 06:40:15 +02:00
set_cuda_devices_for_ci.py Fix Cohere CI (#31263) 2024-06-10 15:16:58 +02:00
slow_documentation_tests.txt Update CodeLlama references (#30218) 2024-05-09 22:57:52 +02:00
sort_auto_mappings.py update ruff version (#30932) 2024-05-22 06:40:15 +02:00
split_doctest_jobs.py Refactor doctest (#30210) 2024-04-15 13:20:36 +02:00
split_model_tests.py consistent job / pytest report / artifact name correspondence (#30392) 2024-04-24 22:32:42 +02:00
tests_fetcher.py Migrate the CI runners to the new clusters (#33849) 2024-10-03 14:39:49 +02:00
update_metadata.py update ruff version (#30932) 2024-05-22 06:40:15 +02:00
update_tiny_models.py Mention model_info.id instead of model_info.modelId (#32106) 2024-07-22 14:14:47 +01:00