transformers/utils
Arthur 474bf508df
Add Segment Anything Model (SAM) (#22654)
* initial commit

* keys match

* update, fix conversion

* fixes, inference working

* fix

* more fixes

* more fixes

* clean up

* more clean up

* fix copies and add convext copied layer norm

* stash

* pretty big upfate

* cleaning

* more cleaning

* fixup stuffs

* fix copies

* fix iinit

* update test removing tokenizer

* nits

* add pretrained

* more nits

* remove tracking of pipeline

* few fixes

* update san and conversion script

* fix mask decoder and prompt encoder conversion

* fixes

* small update

* fix order

* fix

* fix image embeddings

* nites

* few fixes

* fix logits

* clean up

* fixes boxes inference

* v1 AMG

* clean up

* some clean up

* multi points support

* amg working

* fixup

* clean up

* readme

* update toctree

* fix type hint

* multiple fixes

* fixup

* fixes

* updates

* updates

* more tests

* few fixes

* change to `SamForMaskGeneration`

* doc

* fixup

* fix more tests

* multiple fixes

* fix CI tests

* refactor processor

* renamings

* draft the pipeline

* refactor

* fix tests

* fix test

* few cleanings

* fix test

* edit pipelien support chunking

* udate

* add slow tests

* fix nit

* fixup

* fix nit

* current chunk pipleine

* cast boxes in fp32

* nit

* current updates

* piepleine works

* fixup

* clean up config

* fix slow tests

* fix slow tests

* clean up

* update doc and pipeline

* adds more slow tests

* fix slow tests

* cleaning

* tests pass

* add docstring

* fix copies

* clean up

* support batch of images

* style

* dummy is needed, add tests

* fix slow tests

* fix CI

* update

* adds more tests

* fixes

* fixes

* fixup

* fixes

* few fixes

* filter

* few fixes

* some refactor

* touches finales

* fix

* style

* remove pipeline files

* fixes nits

* revert pipeline changes

* fix test

* fixup

* remove automodel for automatic mask generation

* fix failing torch tests

* update mdx

* revert removal of `MODEL_FOR_AUTOMATIC_MASK_GENERATION_MAPPING`

* update sam config based on review

Co-authored-by: amyeroberts <aeroberts4444@gmail.com>
Co-authored-by: sgugger <sylvain.gugger@gmail.com>

* update low_resolution_masks -> pred_masks
inti ln with layer_norm_eps
add_decomposed_rel_pos doc
forward doc of SamForMaskGeneration

* update processor docstring

* remove image processor import empty

* update for testing

* output vision hidden states + clean recomm
also test all iou values

* fixup

* fixup

* remove unused

* Update src/transformers/models/sam/modeling_sam.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/sam/image_processing_sam.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* nits

* fix

* fix CI tests and slow tests

* replace with Amy's processor

* clearer docstring

* add `SamVisionNeck`

* refactor - all CI tests should pass

* fix broken import on Gcolab

* few fixes here and there

* fix another bug

* fix more bugs

* update and merge

* correct ckpt

* address comments

* add tips

* revert

* fix docstring

* replace with `SamModel`

* make fixup

* add support for bathed images and batch ed points

* make fixup this time, really

* make fixup again and again

* few fixes here and there, this should be the touche finale

* Update docs/source/en/model_doc/sam.mdx

* fixup

* correct checkpoints

* correct name

* rm unneeded file

* add notebook

---------

Co-authored-by: younesbelkada <younesbelkada@gmail.com>
Co-authored-by: amyeroberts <aeroberts4444@gmail.com>
Co-authored-by: sgugger <sylvain.gugger@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
2023-04-19 21:01:49 +02:00
..
test_module AutoImageProcessor (#20111) 2022-11-08 19:54:41 +00:00
tf_ops
add_pipeline_model_mapping_to_test.py A script to add/update pipeline_model_mapping systematically (#22180) 2023-04-06 18:08:14 +02:00
check_config_attributes.py [Time-Series] informer model (#21099) 2023-03-07 21:36:38 +01:00
check_config_docstrings.py LLaMA Implementation (#21955) 2023-03-16 09:00:53 -04:00
check_copies.py Add Segment Anything Model (SAM) (#22654) 2023-04-19 21:01:49 +02:00
check_doc_toc.py Apply ruff flake8-comprehensions (#21694) 2023-02-22 09:14:54 +01:00
check_doctest_list.py Update quality tooling for formatting (#21480) 2023-02-06 18:10:56 -05:00
check_dummies.py Cleanup quality (#21493) 2023-02-07 12:27:31 -05:00
check_inits.py Make it easier to develop without a dev install (#22697) 2023-04-11 08:41:53 -04:00
check_model_tester.py Add a new script to check model testers' config (#22063) 2023-03-13 19:11:19 +01:00
check_repo.py Add Segment Anything Model (SAM) (#22654) 2023-04-19 21:01:49 +02:00
check_self_hosted_runner.py Update quality tooling for formatting (#21480) 2023-02-06 18:10:56 -05:00
check_table.py refactor: Make direct_transformers_import util (#21652) 2023-02-16 11:32:32 -05:00
check_task_guides.py Depth estimation task guide (#22205) 2023-03-17 08:36:23 -04:00
check_tf_ops.py
create_dummy_models.py Update tiny model summary file for recent models (#22637) 2023-04-06 22:52:59 +02:00
custom_init_isort.py Update quality tooling for formatting (#21480) 2023-02-06 18:10:56 -05:00
documentation_tests.txt [Pix2struct] Simplify generation (#22527) 2023-04-13 09:01:14 -04:00
download_glue_data.py
extract_warnings.py Make Slack CI reporting stronger (#21823) 2023-02-28 17:12:44 +01:00
get_ci_error_statistics.py Show diff between 2 CI runs on Slack reports (#22798) 2023-04-19 19:27:37 +02:00
get_github_job_time.py Make Slack CI reporting stronger (#21823) 2023-02-28 17:12:44 +01:00
get_modified_files.py exclude deleted files in the fixup script (#21436) 2023-02-03 12:57:02 -05:00
get_previous_daily_ci.py Show diff between 2 CI runs on Slack reports (#22798) 2023-04-19 19:27:37 +02:00
get_test_info.py Add an utility file to get information from test files (#21856) 2023-03-01 17:53:29 +01:00
notification_service_doc_tests.py Update quality tooling for formatting (#21480) 2023-02-06 18:10:56 -05:00
notification_service.py Show diff between 2 CI runs on Slack reports (#22798) 2023-04-19 19:27:37 +02:00
past_ci_versions.py (Re-)Enable Nightly + Past CI (#22393) 2023-03-30 21:06:35 +02:00
prepare_for_doc_test.py Add a check regarding the number of occurrences of ``` (#18389) 2022-08-01 14:23:02 +02:00
print_env.py Print more library versions in CI (#17384) 2022-06-02 10:24:16 +02:00
release.py Clean README in post release job as well. (#17519) 2022-06-02 07:44:03 -04:00
sort_auto_mappings.py Automatically sort auto mappings (#17250) 2022-05-16 13:24:20 -04:00
tests_fetcher.py Mark auto models as important (#22815) 2023-04-17 15:33:01 -04:00
update_metadata.py Add AutoModelForZeroShotImageClassification (#22087) 2023-03-13 12:46:14 +03:00
update_tiny_models.py Update tiny model summary file for recent models (#22637) 2023-04-06 22:52:59 +02:00