transformers/utils
NielsRogge 9c6f7485a6
Add GIT (GenerativeImage2Text) (#20295)
* First draft

* Make model instantiation work

* Fix copied from statement

* More fixes

* Add correct output head

* Improve configuration

* Add conversion script

* Improve conversion script

* Remove token_type_ids

* Fix conversion of projection layers

* Convert all weights

* Use cats image

* Make logits match

* Generate caption on cats image

* Add GITProcessor

* Update conversion script

* Add support for more checkpoints

* Fix conversion script

* Add initial tests

* Remove cross-attention

* More improvements

* Remove is_decoder

* Improve model tests

* Improve tests

* Improve model outputs

* Fix model outputs equivalence

* Fix more tests

* Remove unused code

* Use generate to generate text, no use of cache for now

* Use generate more appropriately

* Fix config tests

* Fix style

* Add support for use_cache

Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>

* Fix style

* Fix GIT vision encoder

* Update README

* Fix integration test

* Set bos and eos token ids

* Improve docs

* Improve code

* Add support for provided attention_mask

* Add copied from statement

* Fix gradient checkpointing test

* Set model_input_names

* Investigate model_input_names

* Remove script

* Fix model inputs

* Fix docstring

* Rename GIT to Git

* Support more models

* Add support for textvqa model

* Add video support

* Extend conversion script for video

* Add support for large variant

* Add support for more models

* Fix config archive map

* Update integration test

* Fix README

* Fix CLIP mean and std

* Update processor

* Fix use_cache for video, thanks @gante

* Remove print statements

* Remove assertion

* Add processor tests

* Fix model_input_names

* Use Auto API for processor

* Fix processor tests

* Fix integration test

* Fix pipeline test

* Make tests faster

* Update conversion script

* Update conversion script

* Convert more checkpoints

* Update conversion script

* Fix typo

* Update docstrings

* Improve code snippets

* Fix doc tests

* Add more code examplesé

* Fix doc tests

* Add integration tests

* Fix unused variable

* revert

* Add GIT to Japanese README

Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-01-03 14:17:18 +01:00
..
test_module AutoImageProcessor (#20111) 2022-11-08 19:54:41 +00:00
tf_ops Check TF ops for ONNX compliance (#10025) 2021-02-15 07:55:10 -05:00
check_config_docstrings.py Create dummy models (#19901) 2022-10-28 13:05:41 +02:00
check_copies.py Add japanese translation of template (#20870) 2022-12-23 14:39:42 +01:00
check_doc_toc.py Split model list on modality (#18328) 2022-08-01 11:10:20 -05:00
check_dummies.py Add some tests for check_dummies (#19146) 2022-09-21 14:54:09 -04:00
check_inits.py Add ESMFold (#19977) 2022-10-31 21:32:58 -04:00
check_repo.py Add GIT (GenerativeImage2Text) (#20295) 2023-01-03 14:17:18 +01:00
check_self_hosted_runner.py Add offline runners info in the Slack report (#19169) 2022-09-23 19:23:05 +02:00
check_table.py Fix some typos. (#17560) 2022-07-11 05:00:13 -04:00
check_tf_ops.py Check TF ops for ONNX compliance (#10025) 2021-02-15 07:55:10 -05:00
create_dummy_models.py [Tiny model creation] deal with ImageProcessor (#20298) 2022-11-17 20:49:46 +01:00
custom_init_isort.py Fix init import_structure sorting (#20477) 2022-11-29 09:46:10 -05:00
documentation_tests.txt Add GIT (GenerativeImage2Text) (#20295) 2023-01-03 14:17:18 +01:00
download_glue_data.py Raise exceptions instead of asserts (#13907) 2021-10-07 12:44:23 +05:30
extract_warnings.py Update some GH action versions (#20537) 2022-12-06 16:54:40 +01:00
get_ci_error_statistics.py Update Past CI report script (#19228) 2022-09-29 19:22:23 +02:00
get_github_job_time.py add a script to get time info. from GA workflow jobs (#18822) 2022-09-01 12:02:52 +02:00
get_modified_files.py Updates the default branch from master to main (#16326) 2022-03-23 03:46:59 -04:00
notification_service_doc_tests.py fix missing block when there is no failure (#18775) 2022-08-29 09:10:13 +02:00
notification_service.py extract warnings in GH workflows (#20487) 2022-11-29 15:58:54 +01:00
past_ci_versions.py Add PyTorch 1.11 to past CI (#18302) 2022-07-26 15:47:23 +02:00
prepare_for_doc_test.py Add a check regarding the number of occurrences of ``` (#18389) 2022-08-01 14:23:02 +02:00
print_env.py Print more library versions in CI (#17384) 2022-06-02 10:24:16 +02:00
release.py Clean README in post release job as well. (#17519) 2022-06-02 07:44:03 -04:00
sort_auto_mappings.py Automatically sort auto mappings (#17250) 2022-05-16 13:24:20 -04:00
tests_fetcher.py Add dpt-hybrid support (#20645) 2022-12-07 17:01:55 +01:00
update_metadata.py Add video classification pipeline (#20151) 2022-12-08 16:22:43 -05:00