Jungwoo Park
44bd590a29
Pix2Struct: fix wrong broadcast axis of attention mask in visual encoder ( #23976 )
...
* fix wrong broadcast axis of attention mask in visual encoder
* fix slow tests
---------
Co-authored-by: younesbelkada <younesbelkada@gmail.com>
2023-06-05 11:47:29 -04:00
Yih-Dar
21741e8c7e
Update test_batched_inference_image_captioning_conditioned ( #23391 )
...
* fix
* fix
* fix test + add more docs
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: younesbelkada <younesbelkada@gmail.com>
2023-05-16 14:49:24 +02:00
NielsRogge
8eb38f638d
[Pix2struct] Simplify generation ( #22527 )
...
* Add model to doc tests
* Remove generate and replace by prepare_inputs_for_generation
* More fixes
* Remove print statements
* Update integration tests
* Fix generate
* Remove model from auto mapping
* Use auto processor
* Fix integration tests
* Fix test
* Add inference code snippet
* Remove is_encoder_decoder
* Update docs
* Remove notebook link
2023-04-13 09:01:14 -04:00
Younes Belkada
b844f8a9ab
[Pix2Struct] Fix slow test ( #22448 )
...
fix slow test
2023-03-29 17:40:45 +02:00
NielsRogge
0e708178ed
[Pix2Struct] Add support to resize embeddings ( #22394 )
...
* First draft
* Fix integration test
* Remove script
* Fix test and typos
* Fix one more test
* Skip tied embeddings test
* Remove line
* Address comments
2023-03-27 11:38:07 -04:00
Younes Belkada
0f68a7f408
Add Pix2Struct ( #21400 )
...
* v1 all keys match
* clean up
* forward pass ok
* add correct image transform
* generate works, logits matching
* clean up
* more refactor
* revert
* revert
* clean up
* clean ups
* clean up
* refactor
* refactor
* fix doc
* fix tokenizer test
* fix toctree
* revert toctree
* oops
* few fixes
* replace to `pixel_embeds`
* make fixup
* test processing & feat extractor
* fix some tests
* more fixes
* make fixup
* clean up
* more clean up
* add a single slow test
* fix test
* make fixup
* fix
* fix authors
* fix toctree
* update docs
* add docstring
* revert change
* Update src/transformers/models/pix2struct/__init__.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* fix tokenizer
* fix processor test
* fix test
* make fixup
* refactor
* fix config
* Update src/transformers/models/pix2struct/image_processing_pix2struct.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* format
* fix
* Update src/transformers/models/pix2struct/image_processing_pix2struct.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* make fixup
* add docstring
* fix issues
* fix
* fix
* fix
* add slow test
* fix
* fix
* fix batched issue
* fix training issues
* fix ci test
* fix slow test
* fix conversion script
* remove unneeded classes
* fix slow test
* fix require backends
* fix masked fill
* revert
* fix softmax
* add large models support
* fix conditional generation
* few fixes
* add instructions
* rm unneeded file
* Update src/transformers/models/pix2struct/convert_pix2struct_original_pytorch_to_hf.py
* fix ci test
* fix ci test really
* Apply suggestions from code review
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* fix nit
* fix nits
* fix image processors nits
* docstring
* clean up
* fix nit
* fix tests
* docstring nit
* fix reshape
* Update src/transformers/models/pix2struct/image_processing_pix2struct.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* fix nit
* fix repetition
* refactor processor
* make patch size consistent
* refactor forward
* fix docstring
* fix max_patches issue
* update docstirng
* update docstring
* fix coped from
* add skip reasons
* few fixes
* Update src/transformers/models/pix2struct/image_processing_pix2struct.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* format
* fix doctests
* refactor and fix
* fix doc build issue
* fix processor test
* small fix conversion script
* replace correct weights
* make fixup
* fix some issues
* Apply suggestions from code review
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* revert config and fixes
* Update src/transformers/models/pix2struct/image_processing_pix2struct.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* more details
* fixes
* fix processor
* fix processor test
* fix
* Apply suggestions from code review
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* make fixup
* fix processor
* Update src/transformers/models/pix2struct/modeling_pix2struct.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* add copied
* make fixup
* fix copies
* update docstring
* refactor
* fix docstring
* fix conversion script
* fix vqa issue
* replace to `flattened_patches`
* nit
* fix numpy issue
* fix image processors
* add batched vqa support
* fix vqa conversion
* make fixup
* fix conversion script
* Apply suggestions from code review
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* make fixup
* add correct docstring
* update docstring
* fix module level + channel dim
* use `make_list_of_images`
* refactor
* correct docstring
* fix authors
* remove `data_format`
* add header text test
* Apply suggestions from code review
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* make fixup
* add checkpoints
---------
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
2023-03-22 16:53:52 +01:00
