omahs
274e79b326
Fix typos ( #37978 )
...
fix typos
2025-05-06 14:45:20 +01:00
cyyever
1e6b546ea6
Use Python 3.9 syntax in tests ( #37343 )
...
Signed-off-by: cyy <cyyever@outlook.com>
2025-04-08 14:12:08 +02:00
Joao Gante
42ebb6c23e
[tests] Parameterized test_eager_matches_sdpa_inference ( #36650 )
2025-03-14 14:41:27 +00:00
co63oc
996f512d52
Fix typos in tests ( #36547 )
...
Signed-off-by: co63oc <co63oc@users.noreply.github.com>
2025-03-05 15:04:06 -08:00
Yoni Gozlan
d7188ba600
Add support for nested images to LLava and VipLLava ( #35558 )
...
* move make_flat_list_of_images and make_batched_videos to image_utils
* remove unnecessary is_vision_available
* move make_nested_list_of_images to image_utils
* fix fast pixtral image processor
* fix import mllama
* fix make_nested_list_of_images
* add tests
* convert 4d arrays/tensors to list
* add test_make_batched_videos
* add support nested batch of videos
* fix image processing qwen2vl
2025-01-30 16:49:20 -05:00
Arthur
b912f5ee43
use torch.testing.assertclose instead to get more details about error in cis ( #35659 )
...
* use torch.testing.assertclose instead to get more details about error in cis
* fix
* style
* test_all
* revert for I bert
* fixes and updates
* more image processing fixes
* more image processors
* fix mamba and co
* style
* less strick
* ok I won't be strict
* skip and be done
* up
2025-01-24 16:55:28 +01:00
Tony Wu
f33a0cebb3
Add ColPali to 🤗 transformers ( #33736 )
...
* feat: run `add-new-model-like`
* feat: add paligemma code with "copied from"
* feat: add ColPaliProcessor
* feat: add ColPaliModel
* feat: add ColPaliConfig
* feat: rename `ColPaliForConditionalGeneration` to `ColPaliModel`
* fixup modeling colpali
* fix: fix root import shortcuts
* fix: fix `modeling_auto` dict
* feat: comment out ColPali test file
* fix: fix typos from `add-new-model-like`
* feat: explicit the forward input args
* feat: move everything to `modular_colpali.py`
* fix: put back ColPaliProcesor
* feat: add auto-generated files
* fix: run `fix-copies`
* fix: remove DOCStRING constants to make modular converter work
* fix: fix typo + modular converter
* fix: add missing imports
* feat: no more errors when loading ColPaliModel
* fix: remove unused args in forward + tweak doc
* feat: rename `ColPaliModel` to `ColPaliForRetrieval`
* fix: apply `fix-copies`
* feat: add ColPaliProcessor to `modular_colpali`
* fix: run make quality + make style
* fix: remove duplicate line in configuration_auto
* feat: make ColPaliModel inehrit from PaliGemmaForConditionalGeneration
* fix: tweak and use ColPaliConfig
* feat: rename `score` to `post_process_retrieval`
* build: run modular formatter + make style
* feat: convert colpali weights + fixes
* feat: remove old weight converter file
* feat: add and validate tests
* feat: replace harcoded path to "vidore/colpali-v1.2-hf" in tests
* fix: add bfloat16 conversion in weight converter
* feat: replace pytest with unittest in modeling colpali test
* feat: add sanity check for weight conversion (doesn't work yet)
* feat: add shape sanity check in weigth converter
* feat: make ColPaliProcessor args explicit
* doc: add doc for ColPali
* fix: trying to fix output mismatch
* feat: tweaks
* fix: ColPaliModelOutput inherits from ModelOutput instead of PaliGemmaCausalLMOutputWithPast
* fix: address comments on PR
* fix: adapt tests to the Hf norm
* wip: try things
* feat: add `__call__` method to `ColPaliProcessor`
* feat: remove need for dummy image in `process_queries`
* build: run new modular converter
* fix: fix incorrect method override
* Fix tests, processing, modular, convert
* fix tokenization auto
* hotfix: manually fix processor -> fixme once convert modular is fixed
* fix: convert weights working
* feat: rename and improve convert weight script
* feat: tweaks
* fest: remove `device` input for `post_process_retrieval`
* refactor: remove unused `get_torch_device`
* Fix all tests
* docs: update ColPali model doc
* wip: fix convert weights to hf
* fix logging modular
* docs: add acknowledgements in model doc
* docs: add missing docstring to ColPaliProcessor
* docs: tweak
* docs: add doc for `ColPaliForRetrievalOutput.forward`
* feat: add modifications from colpali-engine v0.3.2 in ColPaliProcessor
* fix: fix and upload colapli hf weights
* refactor: rename `post_process_retrieval` to `score_retrieval`
* fix: fix wrong typing for `score_retrieval`
* test: add integration test for ColPali
* chore: rerun convert modular
* build: fix root imports
* Update docs/source/en/index.md
Co-authored-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com>
* fix: address PR comments
* wip: reduce the prediction gap in weight conversion
* docs: add comment in weight conversion script
* docs: add example for `ColPaliForRetrieval.forward`
* tests: change dataset path to the new one in hf-internal
* fix: colpali weight conversion works
* test: add fine-grained check for ColPali integration test
* fix: fix typos in convert weight script
* docs: move input docstring in a variable
* fix: remove hardcoded torch device in test
* fix: run the new modular refactor
* docs: fix python example for ColPali
* feat: add option to choose `score_retrieval`'s output dtype and device
* docs: update doc for `score_retrieval`
* feat: add `patch_size` property in ColPali model
* chore: run `make fix-copies`
* docs: update description for ColPali cookbooks
* fix: remove `ignore_index` methods
* feat: remove non-transformers specific methods
* feat: update `__init__.py` to new hf format
* fix: fix root imports in transformers
* feat: remove ColPali's inheritance from PaliGemma
* Fix CI issues
* nit remove prints
* feat: remove ColPali config and model from `modular_colpali.py`
* feat: add `ColPaliPreTrainedModel` and update modeling and configuration code
* fix: fix auto-removed imports in root `__init__.py`
* fix: various fixes
* fix: fix `_init_weight`
* temp: comment `AutoModel.from_config` for experiments
* fix: add missing `output_attentions` arg in ColPali's forward
* fix: fix `resize_token_embeddings`
* fix: make `input_ids` optional in forward
* feat: rename `projection_layer` to `embedding_proj_layer`
* wip: fix convert colpali weight script
* fix tests and convert weights from original repo
* fix unprotected import
* fix unprotected torch import
* fix style
* change vlm_backbone_config to vlm_config
* fix unprotected import in modular this time
* fix: load config from Hub + tweaks in convert weight script
* docs: move example usage from model docstring to model markdown
* docs: fix input docstring for ColPali's forward method
* fix: use `sub_configs` for ColPaliConfig
* fix: remove non-needed sanity checks in weight conversion script + tweaks
* fix: fix issue with `replace_return_docstrings` in ColPali's `forward`
* docs: update docstring for `ColPaliConfig`
* test: change model path in ColPali test
* fix: fix ColPaliConfig
* fix: fix weight conversion script
* test: fix expected weights for ColPali model
* docs: update ColPali markdown
* docs: fix minor typo in ColPaliProcessor
* Fix tests and add _no_split_modules
* add text_config to colpali config
* [run slow] colpali
* move inputs to torch_device in integration test
* skip test_model_parallelism
* docs: clarify quickstart snippet in ColPali's model card
* docs: update ColPali's model card
---------
Co-authored-by: yonigozlan <yoni.gozlan@huggingface.co>
Co-authored-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com>
2024-12-17 11:26:43 +01:00
