Anahita Bhiwandiwalla
3a6e4a221c
Add BridgeTower model ( #20775 )
...
* Commit with BTModel and latest HF code
* Placeholder classes for BTForMLM and BTForITR
* Importing Bert classes from transformers
* Removed objectives.py and dist_utils.py
* Removed swin_transformer.py
* Add image normalization, BridgeTowerForImageAndTextRetrieval
* Add center_crop
* Removing bert tokenizer and LCI references
* Tested config loading from HF transformers hub
* Removed state_dict updates and added path to hub
* Enable center crop
* Getting image_size from config, renaming num_heads and num_layers
* Handling max_length in BridgeTowerProcessor
* Add BridgeTowerForMaskedLM
* Add doc string for BridgeTowerConfig
* Add doc strings for BT config, processor, image processor
* Adding docs, removed swin
* Removed convert_bridgetower_original_to_pytorch.py
* Added doc files for bridgetower, removed is_vision
* Add support attention_mask=None and BridgeTowerModelOutput
* Fix formatting
* Fixes with 'make style', 'make quality', 'make fixup'
* Remove downstream tasks from BridgeTowerModel
* Formatting fixes, add return_dict to BT models
* Clean up after doc_test
* Update BTModelOutput return type, fix todo in doc
* Remove loss_names from init
* implement tests and update tuples returned by models
* Add image reference to bridgetower.mdx
* after make fix-copies, make fixup, make style, make quality, make repo-consistency
* Rename class names with BridgeTower prefix
* Fix for image_size in BTImageProcessor
* implement feature extraction bridgetower tests
* Update image_mean and image_std to be list
* remove unused import
* Removed old comments
* Rework CLIP
* update config in tests followed config update
* Formatting fixes
* Add copied from for BridgeTowerPredictionHeadTransform
* Update bridgetower.mdx
* Update test_feature_extraction_bridgetower.py
* Update bridgetower.mdx
* BridgeTowerForMaskedLM is conditioned on image too
* Add BridgeTowerForMaskedLM
* Fixes
* Call post_init to init weights
* Move freeze layers into method
* Remove BTFeatureExtractor, add BT under multimodal models
* Remove BTFeatureExtractor, add BT under multimodal models
* Code review feedback - cleanup
* Rename variables
* Formatting and style to PR review feedback
* Move center crop after resize
* Use named parameters
* Style fix for modeling_bridgetower.py
* Update docs/source/en/model_doc/bridgetower.mdx
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update docs/source/en/model_doc/bridgetower.mdx
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update docs/source/en/model_doc/bridgetower.mdx
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/bridgetower/modeling_bridgetower.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/bridgetower/modeling_bridgetower.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update docs/source/en/model_doc/bridgetower.mdx
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
* Update src/transformers/models/bridgetower/modeling_bridgetower.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Rename config params, copy BERT classes, clean comments
* Cleanup irtr
* Replace Roberta imports, add BTTextConfig and Model
* Update docs, add visionconfig, consistent arg names
* make fixup
* Comments for forward in BTModel and make fixup
* correct tests
* Remove inconsistent roberta copied from
* Add BridgeTowerTextModel to dummy_pt_objects.py
* Add BridgeTowerTextModel to IGNORE_NON_TESTED
* Update docs for BT Text and Vision Configs
* Treat BridgeTowerTextModel as a private model
* BridgeTowerTextModel as private
* Run make fix-copies
* Adding BTTextModel to PRIVATE_MODELS
* Fix for issue with BT Text and Image configs
* make style changes
* Update README_ja.md
Add から to BridgeTower's description
* Clean up config, .mdx and arg names
* Fix init_weights. Remove nn.Sequential
* Formatting and style fixes
* Re-add tie_word_embeddings in config
* update test implementation
* update style
* remove commented out
* fix style
* Update README with abs for BridgeTower
* fix style
* fix mdx file
* Update bridgetower.mdx
* Update img src in bridgetower.mdx
* Update README.md
* Update README.md
* resolve style failed
* Update _toctree.yml
* Update README_ja.md
* Removed mlp_ratio, rename feats, rename BTCLIPModel
* Replace BTCLIP with BTVisionModel,pass in vision_config to BTVisionModel
* Add test_initialization support
* Add support for output_hidden_states
* Update support for output_hidden_states
* Add support for output_attentions
* Add docstring for output_hidden_states
* update tests
* add bridgetowervisionmodel as private model
* rerun the PR test
* Remove model_type, pass configs to classes, renames
* Change self.device to use weight device
* Remove image_size
* Style check fixes
* Add hidden_size and num_hidden_layers to BridgeTowerTransformer
* Update device setting
* cosmetic update
* trigger test again
* trigger tests again
* Update test_modeling_bridgetower.py
trigger tests again
* Update test_modeling_bridgetower.py
* minor update
* re-trigger tests
* Update docs/source/en/model_doc/bridgetower.mdx
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Remove pad, update max_text_len, doc cleanup, pass eps to LayerNorm
* Added copied to, some more review feedback
* make fixup
* Use BridgeTowerVisionEmbeddings
* Code cleanup
* Fixes for BridgeTowerVisionEmbeddings
* style checks
* re-tests
* fix embedding
* address comment on init file
* retrigger tests
* update import prepare_image_inputs
* update test_image_processing_bridgetower.py to reflect test_image_processing_common.py
* retrigger tests
Co-authored-by: Shaoyen Tseng <shao-yen.tseng@intel.com>
Co-authored-by: Tiep Le <tiep.le@intel.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
Co-authored-by: Tiep Le <97980157+tileintel@users.noreply.github.com>
2023-01-25 14:04:32 -05:00
