transformers/docs/source/en
NielsRogge 9c6f7485a6
Add GIT (GenerativeImage2Text) (#20295)
* First draft

* Make model instantiation work

* Fix copied from statement

* More fixes

* Add correct output head

* Improve configuration

* Add conversion script

* Improve conversion script

* Remove token_type_ids

* Fix conversion of projection layers

* Convert all weights

* Use cats image

* Make logits match

* Generate caption on cats image

* Add GITProcessor

* Update conversion script

* Add support for more checkpoints

* Fix conversion script

* Add initial tests

* Remove cross-attention

* More improvements

* Remove is_decoder

* Improve model tests

* Improve tests

* Improve model outputs

* Fix model outputs equivalence

* Fix more tests

* Remove unused code

* Use generate to generate text, no use of cache for now

* Use generate more appropriately

* Fix config tests

* Fix style

* Add support for use_cache

Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>

* Fix style

* Fix GIT vision encoder

* Update README

* Fix integration test

* Set bos and eos token ids

* Improve docs

* Improve code

* Add support for provided attention_mask

* Add copied from statement

* Fix gradient checkpointing test

* Set model_input_names

* Investigate model_input_names

* Remove script

* Fix model inputs

* Fix docstring

* Rename GIT to Git

* Support more models

* Add support for textvqa model

* Add video support

* Extend conversion script for video

* Add support for large variant

* Add support for more models

* Fix config archive map

* Update integration test

* Fix README

* Fix CLIP mean and std

* Update processor

* Fix use_cache for video, thanks @gante

* Remove print statements

* Remove assertion

* Add processor tests

* Fix model_input_names

* Use Auto API for processor

* Fix processor tests

* Fix integration test

* Fix pipeline test

* Make tests faster

* Update conversion script

* Update conversion script

* Convert more checkpoints

* Update conversion script

* Fix typo

* Update docstrings

* Improve code snippets

* Fix doc tests

* Add more code examplesé

* Fix doc tests

* Add integration tests

* Fix unused variable

* revert

* Add GIT to Japanese README

Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-01-03 14:17:18 +01:00
..
internal MinNewTokensLengthLogitsProcessor for .generate method #20814 (#20892) 2023-01-03 06:29:02 -05:00
main_classes Generate: use GenerationConfig as the basis for .generate() parametrization (#20388) 2022-12-15 18:27:20 +00:00
model_doc Add GIT (GenerativeImage2Text) (#20295) 2023-01-03 14:17:18 +01:00
tasks Spanish translation of asr.mdx and add_new_pipeline.mdx (#20569) 2022-12-12 09:23:23 -05:00
_config.py Enable doc in Spanish (#16518) 2022-04-04 10:25:46 -04:00
_toctree.yml Add GIT (GenerativeImage2Text) (#20295) 2023-01-03 14:17:18 +01:00
accelerate.mdx update to use interlibrary links instead of Markdown (#18500) 2022-08-08 10:53:52 -05:00
add_new_model.mdx fix docs typos in "add_new_model" (#20900) 2022-12-27 02:49:15 -05:00
add_new_pipeline.mdx Spanish translation of asr.mdx and add_new_pipeline.mdx (#20569) 2022-12-12 09:23:23 -05:00
add_tensorflow_model.mdx docs: Resolve many typos in the English docs (#20088) 2022-11-07 09:19:04 -05:00
autoclass_tutorial.mdx Update doc examples feature extractor -> image processor (#20501) 2022-11-30 14:50:55 +00:00
benchmarks.mdx Enable doc in Spanish (#16518) 2022-04-04 10:25:46 -04:00
bertology.mdx Enable doc in Spanish (#16518) 2022-04-04 10:25:46 -04:00
big_models.mdx docs: Resolve many typos in the English docs (#20088) 2022-11-07 09:19:04 -05:00
community.mdx Enable doc in Spanish (#16518) 2022-04-04 10:25:46 -04:00
contributing.md Enable doc in Spanish (#16518) 2022-04-04 10:25:46 -04:00
converting_tensorflow_models.mdx Docs - Guide to add a new TensorFlow model (#19256) 2022-09-30 20:30:38 +01:00
create_a_model.mdx Update doc examples feature extractor -> image processor (#20501) 2022-11-30 14:50:55 +00:00
custom_models.mdx Replace awkward timm link with the expected one (#20109) 2022-11-07 13:57:39 -05:00
debugging.mdx Spanish translation of the file debugging.mdx (#20566) 2022-12-12 10:38:56 -05:00
fast_tokenizers.mdx Enable doc in Spanish (#16518) 2022-04-04 10:25:46 -04:00
glossary.mdx Update doc examples feature extractor -> image processor (#20501) 2022-11-30 14:50:55 +00:00
hpo_train.mdx update doc for perf_train_cpu_many (#19506) 2022-10-11 22:54:19 -04:00
index.mdx Add GIT (GenerativeImage2Text) (#20295) 2023-01-03 14:17:18 +01:00
installation.mdx Move cache folder to huggingface/hub for consistency with hf_hub (#18492) 2022-08-05 13:14:00 -04:00
migration.mdx Enable doc in Spanish (#16518) 2022-04-04 10:25:46 -04:00
model_sharing.mdx Just re-reading the whole doc every couple of months 😬 (#18489) 2022-08-06 09:38:55 +02:00
model_summary.mdx Embed circle packing chart for model summary (#20791) 2022-12-20 10:26:52 -08:00
multilingual.mdx Enable doc in Spanish (#16518) 2022-04-04 10:25:46 -04:00
notebooks.md Enable doc in Spanish (#16518) 2022-04-04 10:25:46 -04:00
pad_truncation.mdx Enable doc in Spanish (#16518) 2022-04-04 10:25:46 -04:00
perf_hardware.mdx [WIP] [doc] performance/scalability revamp (#15723) 2022-05-16 13:36:41 +02:00
perf_infer_cpu.mdx add doc for (#20525) 2022-12-01 16:52:13 +01:00
perf_infer_gpu_many.mdx add doc for (#20525) 2022-12-01 16:52:13 +01:00
perf_infer_gpu_one.mdx add doc for (#20525) 2022-12-01 16:52:13 +01:00
perf_infer_special.mdx Improve performance docs (#17750) 2022-06-23 14:51:54 +02:00
perf_train_cpu_many.mdx update cpu related doc (#20444) 2022-11-28 08:54:35 -05:00
perf_train_cpu.mdx update cpu related doc (#20444) 2022-11-28 08:54:35 -05:00
perf_train_gpu_many.mdx Fix Typo in Docs for GPU (#20509) 2022-11-30 10:41:18 -05:00
perf_train_gpu_one.mdx Migrate torchdynamo to torch.compile (#20634) 2022-12-08 11:18:52 -05:00
perf_train_special.mdx Fix Typo in Docs for GPU (#20509) 2022-11-30 10:41:18 -05:00
perf_train_tpu.mdx Fix Typo in Docs for GPU (#20509) 2022-11-30 10:41:18 -05:00
performance.mdx Fix Typo in Docs for GPU (#20509) 2022-11-30 10:41:18 -05:00
perplexity.mdx Fix incorrect size of input for 1st strided window length in Perplexity of fixed-length models (#18906) 2022-09-06 15:20:12 -04:00
philosophy.mdx Update doc examples feature extractor -> image processor (#20501) 2022-11-30 14:50:55 +00:00
pipeline_tutorial.mdx Fixing the pipeline tutorial test (#20746) 2022-12-13 19:08:30 +01:00
pipeline_webserver.mdx Rework the pipeline tutorial (#20437) 2022-12-06 10:47:31 +01:00
pr_checks.mdx 📝 update documentation build section (#18548) 2022-08-09 18:22:55 -05:00
preprocessing.mdx Fix code sample in preprocess (#20561) 2022-12-05 11:49:43 -08:00
quicktour.mdx typo fix (#20891) 2022-12-26 02:06:23 -05:00
run_scripts.mdx Just re-reading the whole doc every couple of months 😬 (#18489) 2022-08-06 09:38:55 +02:00
sagemaker.mdx Enable doc in Spanish (#16518) 2022-04-04 10:25:46 -04:00
serialization.mdx Add Onnx Config for PoolFormer (#20868) 2022-12-23 01:30:57 -05:00
task_summary.mdx Update doc examples feature extractor -> image processor (#20501) 2022-11-30 14:50:55 +00:00
testing.mdx fixed spelling error in testing.mdx (#20220) 2022-11-15 09:40:06 -05:00
tokenizer_summary.mdx Update tokenizer_summary.mdx (#20135) 2022-11-15 01:18:13 +01:00
torchscript.mdx Breakup export guide (#19271) 2022-10-03 13:18:29 -07:00
training.mdx Convert tokenizer outputs for Keras in doc example (#20732) 2022-12-12 16:14:04 +00:00
troubleshooting.mdx Update doc examples feature extractor -> image processor (#20501) 2022-11-30 14:50:55 +00:00