Pavel Iakubovskii
66291778dd
Refactor Attention implementation for ViT-based models ( #36545 )
...
* Refactor vit attention
* Refactor ViT-based models
* 🚨 🚨 🚨 Fix prefix for DPT
* Update params order
* trigger tests
* Fix Dinov2 attention
* Fix DPT attention impl propagation for backbone config
* Common test fix: config is modif. inplace - avoid it
* view->reshape
* Fixup
* Fixup
* Enable IJepa FA2
* Add FA2 in corresponding model docs
2025-03-20 15:15:01 +00:00
Steven Liu
c0f8d055ce
[docs] Redesign ( #31757 )
...
* toctree
* not-doctested.txt
* collapse sections
* feedback
* update
* rewrite get started sections
* fixes
* fix
* loading models
* fix
* customize models
* share
* fix link
* contribute part 1
* contribute pt 2
* fix toctree
* tokenization pt 1
* Add new model (#32615 )
* v1 - working version
* fix
* fix
* fix
* fix
* rename to correct name
* fix title
* fixup
* rename files
* fix
* add copied from on tests
* rename to `FalconMamba` everywhere and fix bugs
* fix quantization + accelerate
* fix copies
* add `torch.compile` support
* fix tests
* fix tests and add slow tests
* copies on config
* merge the latest changes
* fix tests
* add few lines about instruct
* Apply suggestions from code review
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* fix
* fix tests
---------
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* "to be not" -> "not to be" (#32636 )
* "to be not" -> "not to be"
* Update sam.md
* Update trainer.py
* Update modeling_utils.py
* Update test_modeling_utils.py
* Update test_modeling_utils.py
* fix hfoption tag
* tokenization pt. 2
* image processor
* fix toctree
* backbones
* feature extractor
* fix file name
* processor
* update not-doctested
* update
* make style
* fix toctree
* revision
* make fixup
* fix toctree
* fix
* make style
* fix hfoption tag
* pipeline
* pipeline gradio
* pipeline web server
* add pipeline
* fix toctree
* not-doctested
* prompting
* llm optims
* fix toctree
* fixes
* cache
* text generation
* fix
* chat pipeline
* chat stuff
* xla
* torch.compile
* cpu inference
* toctree
* gpu inference
* agents and tools
* gguf/tiktoken
* finetune
* toctree
* trainer
* trainer pt 2
* optims
* optimizers
* accelerate
* parallelism
* fsdp
* update
* distributed cpu
* hardware training
* gpu training
* gpu training 2
* peft
* distrib debug
* deepspeed 1
* deepspeed 2
* chat toctree
* quant pt 1
* quant pt 2
* fix toctree
* fix
* fix
* quant pt 3
* quant pt 4
* serialization
* torchscript
* scripts
* tpu
* review
* model addition timeline
* modular
* more reviews
* reviews
* fix toctree
* reviews reviews
* continue reviews
* more reviews
* modular transformers
* more review
* zamba2
* fix
* all frameworks
* pytorch
* supported model frameworks
* flashattention
* rm check_table
* not-doctested.txt
* rm check_support_list.py
* feedback
* updates/feedback
* review
* feedback
* fix
* update
* feedback
* updates
* update
---------
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>
2025-03-03 10:33:46 -08:00
Prakarsh Kaushik
293e6271c6
Add sdpa for Vivit ( #33757 )
...
* chore:add sdpa to vivit
* fix:failing slow test_inference_interpolate_pos_encoding(failing on main branch too)
* chore:fix nits
* ci:fix repo consistency failure
* chore:add info and benchmark to model doc
* [run_slow] vivit
* chore:revert interpolation test fix for new issue
* [run_slow] vivit
* [run_slow] vivit
* [run_slow] vivit
* chore:add fallback for output_attentions being True
* [run_slow] vivit
* style:make fixup
* [run_slow] vivit
2024-10-15 11:27:54 +02:00
Maria Khalusova
5964f820db
[Docs] Model_doc structure/clarity improvements ( #26876 )
...
* first batch of structure improvements for model_docs
* second batch of structure improvements for model_docs
* more structure improvements for model_docs
* more structure improvements for model_docs
* structure improvements for cv model_docs
* more structural refactoring
* addressed feedback about image processors
2023-11-03 10:57:03 -04:00
Jegor Kitškerkin
8a5e8a9c2a
Add ViViT ( #22518 )
...
* Add model
* Add ability to get classification head weights
* Add docs
* Add imports to __init__.py
* Run style
* Fix imports and add mdx doc
* Run style
* Fix copyright
* Fix config docstring
* Remove imports of ViViTLayer and load_tf_weights_in_vivit
* Remove FeatureExtractor and replace with ImageProcessor everywhere
* Remove ViViTForPreTraining from vivit.mdx
* Change ViViT -> Vivit everywhere
* Add model_doc to _toctree.yml
* Replace tuples with lists in arguments of VivitConfig
* Rename patch_size to tubelet_size in TubeletEmbeddings
* Fix checkpoint names
* Add tests
* Remove unused num_frames
* Fix imports for VivitImageProcessor
* Minor fixes
* Decrease number of frames in VivitModelTester from 32 to 16
* Decrease number of frames in VivitModelTester from 16 to 8
* Add initialization for pos embeddings
* Rename Vivit -> ViViT in some places
* Fix docstring and formatting
* Rename TubeletEmbeddings -> VivitTubeletEmbeddings
* Remove load_tf_weights_in_vivit
* Change checkpoint name
* Remove Vivit _TOKENIZER_FOR_DOC
* Fix
* Fix VivitTubeletEmbeddings and pass config object as parameter
* Use image_size and num_frames instead of video_size
* Change conversion script and fix differences with the orig implementation
* Fix docstrings
* Add attention head pruning
* Run style and fixup
* Fix tests
* Add ViViT to video_classification.mdx
* Save processor in conversion script
* Fix
* Add image processor test
* Run fixup and style
* Run fix-copies
* Update tests/models/vivit/test_modeling_vivit.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update tests/models/vivit/test_modeling_vivit.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/vivit/modeling_vivit.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Use PyAV instead of decord
* Add unittest.skip
* Run style
* Remove unneeded test
* Update docs/source/en/model_doc/vivit.mdx
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/vivit/configuration_vivit.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/vivit/modeling_vivit.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/vivit/image_processing_vivit.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/vivit/modeling_vivit.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/vivit/modeling_vivit.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/vivit/image_processing_vivit.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/vivit/modeling_vivit.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Add model
* Add docs
* Run style
* Fix imports and add mdx doc
* Remove FeatureExtractor and replace with ImageProcessor everywhere
* Change ViViT -> Vivit everywhere
* Rename Vivit -> ViViT in some places
* Update src/transformers/models/vivit/image_processing_vivit.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Run make style
* Remove inputs save
* Fix image processor
* Fix
* Run `make style`
* Decrease parameters of VivitModelTester
* Decrease tubelet size
* Rename vivit.mdx
* Update src/transformers/models/vivit/image_processing_vivit.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/vivit/image_processing_vivit.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/vivit/image_processing_vivit.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Fix default values in image_processing_vivit.py
---------
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
2023-07-11 14:04:04 +01:00
