transformers/docs/source
NielsRogge 65b20b739b
Add Perceiver IO (#14487)
* First draft

* Style and remove mlm

* Make forward pass work

* More improvements

* More improvements

* Fix bug

* More improvements

* More improvements

* Add PerceiverTokenizer first draft

* Improve conversion script

* More improvements

* Make conversion script work for the encoder

* Make conversion script work with local pickle files

* Style & quality, fix-copies

* Add dummy input to conversion script

* Add absolute position embeddings to TextPreProcessor

* Make forward pass of encoder work

* More improvements

* Move text preprocessor to separate script

* More improvements

* More improvements

* Add post processor

* Make MLM model work

* Style

* Add PerceiverForMaskedLM

* Add PerceiverImagePreprocessor

* Make style

* Make PerceiverForImageClassification work

* More improvements

* More improvements

* Use tokenizer in conversion script

* Use PerceiverForMaskedLM in conversion script

* Define custom PerceiverModelOutput

* Improve PerceiverAttention to make it work for both MLM and image classification

* More improvements

* More improvements

* More improvements to the conversion script

* Make conversion script work for both MLM and image classification

* Add PerceiverFeatureExtractor

* More improvements

* Style and quality

* Add center cropping

* Fix bug

* Small fix

* Add print statement

* Fix bug in image preprocessor

* Fix bug with conversion script

* Make output position embeddings an nn.Parameter layer instead of nn.Embedding

* Comment out print statements

* Add position encoding classes

* More improvements

* Use position_encoding_kwargs

* Add PerceiverForImageClassificationFourier

* Make style & quality

* Add PerceiverForImageClassificationConvProcessing

* Style & quality

* Add flow model

* Move processors to modeling file

* Make position encodings modular

* Make basic decoder use modular position encodings

* Add PerceiverForOpticalFlow to conversion script

* Add AudioPreprocessor

* Make it possible for the basic decoder to use Fourier position embeddings

* Add PerceiverForMultimodalAutoencoding

* Improve model for optical flow

* Improve _build_network_inputs method

* Add print statement

* Fix device issue

* Fix device of Fourier embeddings

* Add print statements for debugging

* Add another print statement

* Add another print statement

* Add another print statement

* Add another print statement

* Improve PerceiverAudioPreprocessor

* Improve conversion script for multimodal modal

* More improvements

* More improvements

* Improve multimodal model

* Make forward pass multimodal model work

* More improvements

* Improve tests

* Fix some more tests

* Add output dataclasses

* Make more tests pass

* Add print statements for debuggin

* Add tests for image classification

* Add PerceiverClassifierOutput

* More improvements

* Make more tests pass for the optical flow model

* Make style & quality

* Small improvements

* Don't support training for optical flow model for now

* Fix _prepare_for_class for tests

* Make more tests pass, add some docs

* Add multimodal model to tests

* Minor fixes

* Fix tests

* Improve conversion script

* Make fixup

* Remove pos_dim argument

* Fix device issue

* Potential fix for OOM

* Revert previous commit

* Fix test_initialization

* Add print statements for debugging

* Fix print statement

* Add print statement

* Add print statement

* Add print statement

* Add print statement

* Add print statement

* Add print statement

* Remove need for output_shape

* Comment out output_shape

* Remove unnecessary code

* Improve docs

* Fix make fixup

* Remove PerceiverTextProcessor from init

* Improve docs

* Small improvement

* Apply first batch of suggestions from code review

* Apply more suggestions from code review

* Update docstrings

* Define dicts beforehand for readability

* Rename task to architecture in conversion script, include PerceiverModel in tests

* Add print statements for debugging

* Fix tests on GPU

* Remove preprocessors, postprocessors and decoders from main init

* Add integration test

* Fix docs

* Replace einops by torch

* Update for new docs frontend

* Rename PerceiverForImageClassification

* Improve docs

* Improve docs

* Improve docs of PerceiverModel

* Fix some more tests

* Improve center_crop

* Add PerceiverForSequenceClassification

* Small improvements

* Fix tests

* Add integration test for optical flow model

* Clean up

* Add tests for tokenizer

* Fix tokenizer by adding special tokens properly

* Fix CI
2021-12-08 14:20:34 +01:00
..
imgs [doc] bf16/tf32 guide (#14579) 2021-12-01 14:18:58 -08:00
internal Fix doc building error 2021-08-12 05:49:02 -04:00
main_classes Add mLUKE (#14640) 2021-12-07 00:25:28 -05:00
model_doc Add Perceiver IO (#14487) 2021-12-08 14:20:34 +01:00
_toctree.yml Rename toctree.yml -> _toctree.yml (#14594) 2021-12-02 08:58:39 +01:00
add_new_model.rst Update doc img links (#14593) 2021-12-02 09:01:35 +01:00
add_new_pipeline.rst Improve a add-new-pipeline docs a bit (#14485) 2021-11-22 10:35:49 -05:00
benchmarks.rst [Docs] fixed broken link (#12205) 2021-06-16 15:14:53 -04:00
bertology.rst Fix documentation links always pointing to master. (#9217) 2021-01-05 06:18:48 -05:00
community.md Doc new front (#14590) 2021-12-01 14:13:02 -05:00
contributing.md Update installation page and add contributing to the doc (#5084) 2020-06-17 14:01:10 -04:00
converting_tensorflow_models.rst Doc new front (#14590) 2021-12-01 14:13:02 -05:00
custom_datasets.rst Rewrite guides for fine-tuning with Datasets (#13923) 2021-11-09 14:12:50 -05:00
debugging.rst Fix some writing issues in the docs (#14136) 2021-10-25 07:48:02 -04:00
examples.md per_device instead of per_gpu/error thrown when argument unknown (#4618) 2020-05-27 11:36:55 -04:00
fast_tokenizers.rst Documentation about loading a fast tokenizer within Transformers (#11029) 2021-04-05 10:51:16 -04:00
glossary.rst Add video links to the documentation (#12162) 2021-06-15 06:37:37 -04:00
index.mdx Add Perceiver IO (#14487) 2021-12-08 14:20:34 +01:00
index.rst Add Perceiver IO (#14487) 2021-12-08 14:20:34 +01:00
installation.md Fix some typos in the docs (#14126) 2021-10-25 07:40:44 -04:00
migration.md Doc new front (#14590) 2021-12-01 14:13:02 -05:00
model_sharing.rst Fix some writing issues in the docs (#14136) 2021-10-25 07:48:02 -04:00
model_summary.rst Update doc img links (#14593) 2021-12-02 09:01:35 +01:00
multilingual.rst Add mLUKE (#14640) 2021-12-07 00:25:28 -05:00
notebooks.md Update notebooks (#3620) 2020-04-06 14:32:39 -04:00
parallelism.md Update doc img links (#14593) 2021-12-02 09:01:35 +01:00
performance.md [trainer] add tf32-mode control (#14606) 2021-12-03 10:08:58 -08:00
perplexity.rst Update doc img links (#14593) 2021-12-02 09:01:35 +01:00
philosophy.rst Minor documentation revisions from copyediting (#9266) 2020-12-23 10:15:49 -05:00
pr_checks.md Quality explain (#14264) 2021-11-03 17:43:19 -04:00
preprocessing.rst Doc new front (#14590) 2021-12-01 14:13:02 -05:00
quicktour.rst Doc new front (#14590) 2021-12-01 14:13:02 -05:00
sagemaker.md remove documentation (#12657) 2021-07-12 18:02:51 +02:00
serialization.rst Add Camembert to models exportable with ONNX (#14059) 2021-10-26 11:22:22 +02:00
task_summary.rst Fix broken link in translation section (#14087) 2021-10-20 15:10:57 -04:00
testing.rst [testing] auto-replay captured streams (#13803) 2021-09-30 09:26:49 -07:00
tokenizer_summary.rst Fix some typos in the docs (#14126) 2021-10-25 07:40:44 -04:00
training.rst Doc new front (#14590) 2021-12-01 14:13:02 -05:00
troubleshooting.md Doc new front (#14590) 2021-12-01 14:13:02 -05:00