transformers/docs/source/en
Arthur 474bf508df
Add Segment Anything Model (SAM) (#22654)
* initial commit

* keys match

* update, fix conversion

* fixes, inference working

* fix

* more fixes

* more fixes

* clean up

* more clean up

* fix copies and add convext copied layer norm

* stash

* pretty big upfate

* cleaning

* more cleaning

* fixup stuffs

* fix copies

* fix iinit

* update test removing tokenizer

* nits

* add pretrained

* more nits

* remove tracking of pipeline

* few fixes

* update san and conversion script

* fix mask decoder and prompt encoder conversion

* fixes

* small update

* fix order

* fix

* fix image embeddings

* nites

* few fixes

* fix logits

* clean up

* fixes boxes inference

* v1 AMG

* clean up

* some clean up

* multi points support

* amg working

* fixup

* clean up

* readme

* update toctree

* fix type hint

* multiple fixes

* fixup

* fixes

* updates

* updates

* more tests

* few fixes

* change to `SamForMaskGeneration`

* doc

* fixup

* fix more tests

* multiple fixes

* fix CI tests

* refactor processor

* renamings

* draft the pipeline

* refactor

* fix tests

* fix test

* few cleanings

* fix test

* edit pipelien support chunking

* udate

* add slow tests

* fix nit

* fixup

* fix nit

* current chunk pipleine

* cast boxes in fp32

* nit

* current updates

* piepleine works

* fixup

* clean up config

* fix slow tests

* fix slow tests

* clean up

* update doc and pipeline

* adds more slow tests

* fix slow tests

* cleaning

* tests pass

* add docstring

* fix copies

* clean up

* support batch of images

* style

* dummy is needed, add tests

* fix slow tests

* fix CI

* update

* adds more tests

* fixes

* fixes

* fixup

* fixes

* few fixes

* filter

* few fixes

* some refactor

* touches finales

* fix

* style

* remove pipeline files

* fixes nits

* revert pipeline changes

* fix test

* fixup

* remove automodel for automatic mask generation

* fix failing torch tests

* update mdx

* revert removal of `MODEL_FOR_AUTOMATIC_MASK_GENERATION_MAPPING`

* update sam config based on review

Co-authored-by: amyeroberts <aeroberts4444@gmail.com>
Co-authored-by: sgugger <sylvain.gugger@gmail.com>

* update low_resolution_masks -> pred_masks
inti ln with layer_norm_eps
add_decomposed_rel_pos doc
forward doc of SamForMaskGeneration

* update processor docstring

* remove image processor import empty

* update for testing

* output vision hidden states + clean recomm
also test all iou values

* fixup

* fixup

* remove unused

* Update src/transformers/models/sam/modeling_sam.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/sam/image_processing_sam.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* nits

* fix

* fix CI tests and slow tests

* replace with Amy's processor

* clearer docstring

* add `SamVisionNeck`

* refactor - all CI tests should pass

* fix broken import on Gcolab

* few fixes here and there

* fix another bug

* fix more bugs

* update and merge

* correct ckpt

* address comments

* add tips

* revert

* fix docstring

* replace with `SamModel`

* make fixup

* add support for bathed images and batch ed points

* make fixup this time, really

* make fixup again and again

* few fixes here and there, this should be the touche finale

* Update docs/source/en/model_doc/sam.mdx

* fixup

* correct checkpoints

* correct name

* rm unneeded file

* add notebook

---------

Co-authored-by: younesbelkada <younesbelkada@gmail.com>
Co-authored-by: amyeroberts <aeroberts4444@gmail.com>
Co-authored-by: sgugger <sylvain.gugger@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
2023-04-19 21:01:49 +02:00
..
internal Generate: TextIteratorStreamer (streamer for gradio) (#22501) 2023-04-03 15:04:37 +01:00
main_classes [bnb] Let's make serialization of int8 models possible (#22177) 2023-04-12 08:01:18 -04:00
model_doc Add Segment Anything Model (SAM) (#22654) 2023-04-19 21:01:49 +02:00
tasks Fix word_ids hyperlink (#22765) 2023-04-14 16:18:15 +01:00
_config.py Enable doc in Spanish (#16518) 2022-04-04 10:25:46 -04:00
_toctree.yml Add Segment Anything Model (SAM) (#22654) 2023-04-19 21:01:49 +02:00
accelerate.mdx update to use interlibrary links instead of Markdown (#18500) 2022-08-08 10:53:52 -05:00
add_new_model.mdx Make it easier to develop without a dev install (#22697) 2023-04-11 08:41:53 -04:00
add_new_pipeline.mdx Spanish translation of asr.mdx and add_new_pipeline.mdx (#20569) 2022-12-12 09:23:23 -05:00
add_tensorflow_model.mdx Make it easier to develop without a dev install (#22697) 2023-04-11 08:41:53 -04:00
attention.mdx Refactor model summary (#21408) 2023-02-15 10:35:14 -08:00
autoclass_tutorial.mdx Update doc examples feature extractor -> image processor (#20501) 2022-11-30 14:50:55 +00:00
benchmarks.mdx Enable doc in Spanish (#16518) 2022-04-04 10:25:46 -04:00
bertology.mdx update: bertology paper (#22012) 2023-03-08 07:54:30 -05:00
big_models.mdx docs: Resolve many typos in the English docs (#20088) 2022-11-07 09:19:04 -05:00
community.mdx Fix en documentation typos (#21799) 2023-02-27 08:36:36 +01:00
contributing.md Enable doc in Spanish (#16518) 2022-04-04 10:25:46 -04:00
converting_tensorflow_models.mdx Docs - Guide to add a new TensorFlow model (#19256) 2022-09-30 20:30:38 +01:00
create_a_model.mdx Documentation code sample fixes (#21302) 2023-01-25 11:33:39 -05:00
custom_models.mdx Replace awkward timm link with the expected one (#20109) 2022-11-07 13:57:39 -05:00
debugging.mdx Spanish translation of the file debugging.mdx (#20566) 2022-12-12 10:38:56 -05:00
fast_tokenizers.mdx Enable doc in Spanish (#16518) 2022-04-04 10:25:46 -04:00
generation_strategies.mdx Generate: Add assisted generation (#22211) 2023-04-18 17:36:56 +01:00
glossary.mdx docs: New terms and updates to glossary (#21982) 2023-03-13 19:09:37 -04:00
hpo_train.mdx update doc for perf_train_cpu_many (#19506) 2022-10-11 22:54:19 -04:00
index.mdx Add Segment Anything Model (SAM) (#22654) 2023-04-19 21:01:49 +02:00
installation.mdx Can't install tf2 on M1 Chip by default (#22046) 2023-03-09 07:44:58 -05:00
migration.mdx Enable doc in Spanish (#16518) 2022-04-04 10:25:46 -04:00
model_sharing.mdx Fix PushToHubCallback import in Share a model docs (#21457) 2023-02-06 09:26:22 -05:00
model_summary.mdx Refactor model summary (#21408) 2023-02-15 10:35:14 -08:00
multilingual.mdx Enable doc in Spanish (#16518) 2022-04-04 10:25:46 -04:00
notebooks.md Enable doc in Spanish (#16518) 2022-04-04 10:25:46 -04:00
pad_truncation.mdx Example of pad_to_multiple_of for padding and truncation guide & docstring update (#22278) 2023-03-20 14:18:55 -04:00
perf_hardware.mdx [WIP] [doc] performance/scalability revamp (#15723) 2022-05-16 13:36:41 +02:00
perf_infer_cpu.mdx add doc for (#20525) 2022-12-01 16:52:13 +01:00
perf_infer_gpu_many.mdx add doc for (#20525) 2022-12-01 16:52:13 +01:00
perf_infer_gpu_one.mdx [Doc] Fix int8 docs (#21487) 2023-02-07 15:09:27 +01:00
perf_infer_special.mdx Improve performance docs (#17750) 2022-06-23 14:51:54 +02:00
perf_train_cpu_many.mdx update cpu related doc (#20444) 2022-11-28 08:54:35 -05:00
perf_train_cpu.mdx Add perf numbers for perf_train_cpu (#20974) 2023-02-06 09:20:43 -05:00
perf_train_gpu_many.mdx Fix Typo in Docs for GPU (#20509) 2022-11-30 10:41:18 -05:00
perf_train_gpu_one.mdx Migrate torchdynamo to torch.compile (#20634) 2022-12-08 11:18:52 -05:00
perf_train_special.mdx Fix Typo in Docs for GPU (#20509) 2022-11-30 10:41:18 -05:00
perf_train_tpu_tf.mdx Typos/fixes to link syntax (#21450) 2023-02-07 15:19:19 +00:00
perf_train_tpu.mdx Fix Typo in Docs for GPU (#20509) 2022-11-30 10:41:18 -05:00
performance.mdx Fix Typo in Docs for GPU (#20509) 2022-11-30 10:41:18 -05:00
perplexity.mdx Fix bug in perplexity guide calculations and update perplexity numbers. Fixes #22348 (#22411) 2023-03-28 09:09:17 -04:00
philosophy.mdx Update doc examples feature extractor -> image processor (#20501) 2022-11-30 14:50:55 +00:00
pipeline_tutorial.mdx Modify pipeline_tutorial.mdx (#22726) 2023-04-12 15:20:25 +01:00
pipeline_webserver.mdx Update quality tooling for formatting (#21480) 2023-02-06 18:10:56 -05:00
pr_checks.mdx Make it easier to develop without a dev install (#22697) 2023-04-11 08:41:53 -04:00
preprocessing.mdx Updates to computer vision section of the Preprocess doc (#21181) 2023-01-19 08:43:36 -05:00
quicktour.mdx Fix 2 quicktour file doctest (#21742) 2023-02-23 09:41:28 +01:00
run_scripts.mdx Just re-reading the whole doc every couple of months 😬 (#18489) 2022-08-06 09:38:55 +02:00
sagemaker.mdx Enable doc in Spanish (#16518) 2022-04-04 10:25:46 -04:00
serialization.mdx Add Mega: Moving Average Equipped Gated Attention (#21766) 2023-03-24 08:17:27 -04:00
task_summary.mdx Remove trailing 'extractive' word from en documentation (#21594) 2023-02-13 10:09:00 -05:00
tasks_explained.mdx Update task summary (#21067) 2023-02-02 11:41:27 -08:00
testing.mdx [tests] add accelerate marker (#21743) 2023-02-27 12:33:34 +01:00
tf_xla.mdx Rewrite a couple of lines in the TF XLA doc (#21177) 2023-01-18 17:53:05 +00:00
tokenizer_summary.mdx Update tokenizer_summary.mdx (#20135) 2022-11-15 01:18:13 +01:00
torchscript.mdx Breakup export guide (#19271) 2022-10-03 13:18:29 -07:00
training.mdx Fix code example in training tutorial (#21201) 2023-01-20 07:38:15 -08:00
troubleshooting.mdx Removed BLIP mention from the troubleshooting guide (#21872) 2023-03-01 08:26:25 -05:00