Yih-Dar
397720fb14
Skip a failing test on main for now ( #22911 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-04-21 10:22:54 +02:00
SUSHMANTH REDDY
8a817e1eca
moved labels to the same device as logits for LILT model ( #22898 )
2023-04-20 14:49:47 -04:00
Roy Hvaara
515d6a551e
[tensorflow] Add support for the is_symbolic_tensor predicate ( #22878 )
...
This predicate will become available in tensorflow starting with version
2.14.
Co-authored-by: Russell Power <power@google.com>
2023-04-20 19:46:42 +01:00
Zachary Mueller
5764e67cee
Revert DeepSpeed stuff from accelerate integration ( #22899 )
2023-04-20 14:23:59 -04:00
Arthur
f143037789
Add automatic-mask-generation pipeline for Segment Anything Model (SAM) ( #22840 )
...
* cleanup
* updates
* more refactoring
* make style
* update inits
* support other inputs in base
* update based on review
Co-authored-by: Nicolas Patry <patry.nicolas@gmail.com>
* Update tests/pipelines/test_pipelines_automatic_mask_generation.py
Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com>
* update
* fixup
* TODO x and y to refactor, _h _w refactored here
* update docstring
* more nits
* style on these
* more doc fix
* rename variables
* update
* updates
* style
* update
* fix `_mask_to_rle_pytorch`
* styling
* fix ask to rle, wrong outputs
* add device arg
* update
* more updates, fix tets
* udpate
* update docstrings
* styling
* fixup
* add notebook on the docs
* update orginal sizes
* fix docstring
* updat condition on point_per-batch
* updates tests
* fix CI test
* extend is required, append does not work!
* fixup
* fix CI tests
* whit pixels left
* address doc comments
* fix doc
* slow pipeline tests
* update auto init
* add revision
* make fixup
* update p!ipoeline tag when calling tests
* alphabeitcal order in inits
* fix copies
* last style nits
* Apply suggestions from code review
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* reformat docstring
* more reformat
* address most of the comments
* Update src/transformers/pipelines/mask_generation.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* final refactor
* Update src/transformers/models/sam/image_processing_sam.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* fixup and fix slow tests
* revert
---------
Co-authored-by: Nicolas Patry <patry.nicolas@gmail.com>
Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com>
Co-authored-by: younesbelkada <younesbelkada@gmail.com>
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
2023-04-20 19:27:24 +02:00
amyeroberts
e5f3487190
Pin flax & optax version ( #22895 )
...
* Pin optax version
* Pin flax too
* Fixup
2023-04-20 17:30:14 +01:00
Matt
6dc0a849b7
Fix weight tying in TF-ESM ( #22839 )
...
Fix weight tying in ESM
2023-04-20 15:50:31 +01:00
Aashiq Muhamed
3b61d2890d
Include decoder_attention_mask in T5 model inputs ( #22835 )
2023-04-20 15:05:36 +01:00
SUSHMANTH REDDY
91d6a593f1
moved labels to the same device as logits for OTP, CODEGEN ,gptj and pixel2struct model ( #22872 )
...
* moved labels to the same device as logits for OTP model
* moved labels to the same device as logits for CODEGEN model
* Update modeling_codegen.py
* moved labels to the same device as logits for gptj and pix2struct model
* Update modeling_pix2struct.py
2023-04-20 08:52:54 -04:00
Sayak Paul
4116d1ec75
[Examples/TensorFlow] minor refactoring to allow compatible datasets to work ( #22879 )
...
minor refactoring to allow compatible datasets to work.
2023-04-20 18:21:01 +05:30
Younes Belkada
10dd3a7d1c
[SAM] Change to facebook/sam-vit-base ( #22891 )
...
change to `facebook/sam-vit-base`
2023-04-20 14:11:18 +02:00
Quentin Ambard
aa43a76538
fix warning function call creating logger error (max_length and max_new_tokens) ( #22889 )
2023-04-20 13:08:03 +01:00
Yih-Dar
aa4316757d
Change schedule CI time ( #22884 )
...
* fix
* Update .github/workflows/self-nightly-past-ci-caller.yml
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
2023-04-20 14:01:08 +02:00
xloem
d50db469c0
Generation: only search for eos_token if set ( #22875 )
...
Generation: only check for eos_token if set
The check for unfinished_sequences.max(), which is to find sequences
that have ended early via eos_token_id, creates a synchronization point
even when there is no eos_token, which slows inference down.
This change moves the calculation to inside the condition checking for
eos_token, so that such slowdown may be removed by disabling this token.
Co-authored-by: John Doe <john.doe@example.com>
2023-04-20 12:18:28 +01:00
Oscar
a438a0941c
fix: Correct small typo in docstring ( #22857 )
...
* fix: Correct small typo in docstring
* fix: Run make fixup
2023-04-20 11:58:52 +01:00
fxmarty
4cfe328bae
Fix SAM example in documentation ( #22887 )
...
fix sam example
2023-04-20 12:22:42 +02:00
Shanmugam Ramasamy
cb47293eba
Patching clip model to create mask tensor on the device ( #22711 )
...
* Patching clip model to create mask tensor on the device
* Addressing PR's comments
* Addressing PR's comments
* Addressing PR's comments
---------
Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com>
2023-04-20 10:58:52 +01:00
Younes Belkada
2da73f6302
[SAM] Correct arxiv link ( #22886 )
...
put correct link
2023-04-20 11:23:12 +02:00
Joao Gante
4060d6857e
XGLM: Fix left-padding (PT and TF) ( #22828 )
2023-04-20 10:01:56 +01:00
Arthur
474bf508df
Add Segment Anything Model (SAM) ( #22654 )
...
* initial commit
* keys match
* update, fix conversion
* fixes, inference working
* fix
* more fixes
* more fixes
* clean up
* more clean up
* fix copies and add convext copied layer norm
* stash
* pretty big upfate
* cleaning
* more cleaning
* fixup stuffs
* fix copies
* fix iinit
* update test removing tokenizer
* nits
* add pretrained
* more nits
* remove tracking of pipeline
* few fixes
* update san and conversion script
* fix mask decoder and prompt encoder conversion
* fixes
* small update
* fix order
* fix
* fix image embeddings
* nites
* few fixes
* fix logits
* clean up
* fixes boxes inference
* v1 AMG
* clean up
* some clean up
* multi points support
* amg working
* fixup
* clean up
* readme
* update toctree
* fix type hint
* multiple fixes
* fixup
* fixes
* updates
* updates
* more tests
* few fixes
* change to `SamForMaskGeneration`
* doc
* fixup
* fix more tests
* multiple fixes
* fix CI tests
* refactor processor
* renamings
* draft the pipeline
* refactor
* fix tests
* fix test
* few cleanings
* fix test
* edit pipelien support chunking
* udate
* add slow tests
* fix nit
* fixup
* fix nit
* current chunk pipleine
* cast boxes in fp32
* nit
* current updates
* piepleine works
* fixup
* clean up config
* fix slow tests
* fix slow tests
* clean up
* update doc and pipeline
* adds more slow tests
* fix slow tests
* cleaning
* tests pass
* add docstring
* fix copies
* clean up
* support batch of images
* style
* dummy is needed, add tests
* fix slow tests
* fix CI
* update
* adds more tests
* fixes
* fixes
* fixup
* fixes
* few fixes
* filter
* few fixes
* some refactor
* touches finales
* fix
* style
* remove pipeline files
* fixes nits
* revert pipeline changes
* fix test
* fixup
* remove automodel for automatic mask generation
* fix failing torch tests
* update mdx
* revert removal of `MODEL_FOR_AUTOMATIC_MASK_GENERATION_MAPPING`
* update sam config based on review
Co-authored-by: amyeroberts <aeroberts4444@gmail.com>
Co-authored-by: sgugger <sylvain.gugger@gmail.com>
* update low_resolution_masks -> pred_masks
inti ln with layer_norm_eps
add_decomposed_rel_pos doc
forward doc of SamForMaskGeneration
* update processor docstring
* remove image processor import empty
* update for testing
* output vision hidden states + clean recomm
also test all iou values
* fixup
* fixup
* remove unused
* Update src/transformers/models/sam/modeling_sam.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/models/sam/image_processing_sam.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* nits
* fix
* fix CI tests and slow tests
* replace with Amy's processor
* clearer docstring
* add `SamVisionNeck`
* refactor - all CI tests should pass
* fix broken import on Gcolab
* few fixes here and there
* fix another bug
* fix more bugs
* update and merge
* correct ckpt
* address comments
* add tips
* revert
* fix docstring
* replace with `SamModel`
* make fixup
* add support for bathed images and batch ed points
* make fixup this time, really
* make fixup again and again
* few fixes here and there, this should be the touche finale
* Update docs/source/en/model_doc/sam.mdx
* fixup
* correct checkpoints
* correct name
* rm unneeded file
* add notebook
---------
Co-authored-by: younesbelkada <younesbelkada@gmail.com>
Co-authored-by: amyeroberts <aeroberts4444@gmail.com>
Co-authored-by: sgugger <sylvain.gugger@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
2023-04-19 21:01:49 +02:00
Matt
898efca72a
Fix to removing ESM special tokens ( #22870 )
...
Fix to make sure the EOS token doesn't come back
2023-04-19 19:42:29 +01:00
Zachary Mueller
a8aad0ec93
Fixup multigpu local_rank ( #22869 )
...
Fixup multigpu tests
2023-04-19 14:37:16 -04:00
Yih-Dar
06bab00338
Remove some pipeline skip cases ( #22865 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-04-19 20:27:19 +02:00
Yih-Dar
648bd5a8aa
Show diff between 2 CI runs on Slack reports ( #22798 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-04-19 19:27:37 +02:00
amyeroberts
5f97bbc124
Remove 'main' from doc links ( #22860 )
2023-04-19 15:03:57 +01:00
Yih-Dar
4603fe9b1f
use accelerate@main in CI ( #22859 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-04-19 14:58:53 +02:00
Elabonga Atuo
337225ec1c
feat(model parallelism): move labels to the same device as logits for M2M100 ( #22850 )
...
moved logits for m2m_100
2023-04-19 08:54:27 -04:00
Liu Chenyang
6bd8ae2640
move preprocess_logits_for_metrics before _nested_gather in trainer.e… ( #22603 )
...
* move preprocess_logits_for_metrics before _nested_gather in trainer.evaluation_loop
* fix
* Update src/transformers/trainer.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* fix
* fix
---------
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-04-19 08:53:47 -04:00
Matthijs Hollemans
c582e8aad0
fix SpeechT5 doc comments ( #22854 )
...
fix doc comments
2023-04-19 14:10:40 +02:00
Youssef Adarrab
84a6570e7b
Make ClipSeg compatible with model parallelism ( #22844 )
2023-04-18 19:31:59 -04:00
Zachary Mueller
5bb4ec6233
Raise err if minimum Accelerate version isn't available ( #22841 )
...
* Add warning about accelerate
* Version block Accelerate
* Include parse
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Check partial state
* Update param
---------
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-04-18 14:25:02 -04:00
Sylvain Gugger
5f09219400
Fix from_pretrained when model is instantiated on the meta device ( #22837 )
2023-04-18 13:54:18 -04:00
Sylvain Gugger
5f9b825c89
Use code on the Hub from another repo ( #22814 )
...
* initial work
* Add other classes
* Refactor code
* Move warning and fix dynamic pipeline
* Issue warning when necessary
* Add test
* Do not skip auto tests
* Fix failing tests
* Refactor and address review comments
* Address review comments
2023-04-18 13:46:11 -04:00
Zachary Mueller
aec10d162f
Update accelerate version + warning check fix ( #22833 )
2023-04-18 12:51:32 -04:00
Joao Gante
78cda46f17
Generate: Add assisted generation ( #22211 )
...
* working mvp
* remove breakpoint
* fix commit
* standardize outputs
* tmp commit
* tests almost ready
* tmp commit
* skip a few models
* Add streaming; Docs and examples
* document limitations
* PR commits
* Amy PR comments
2023-04-18 17:36:56 +01:00
Yih-Dar
90247d3e01
Fix test_eos_token_id_int_and_list_top_k_top_sampling ( #22826 )
...
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-04-18 16:04:51 +02:00
Yih-Dar
1ebc1dee92
Fix Past CI not running against the latest main ( #22823 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-04-18 15:41:41 +02:00
Gabriel Yang
42288269c3
🌐 [i18n-KO] Fix anchor links for docs auto_tutorial, training ( #22796 )
...
docs: ko: fix anchor links for docs (auto_tutorial, training)
Co-authored-by: Hyeonseo Yun <0525_hhgus@naver.com>
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-authored-by: Na Yeon Han <nayeon2.han@gmail.com>
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>
2023-04-18 09:11:30 -04:00
Matthijs Hollemans
ac2bc50a10
TTS fine-tuning for SpeechT5 ( #21824 )
...
* wrong argument name
* append eos_token_id
* all tokenizers need mask and ctc_blank tokens
* remove reduction factor from feature extractor
* add proper TTS loss
* did shifting the wrong way around
* mask out padded portions
* remove logits again (don't really need it)
* fix unit tests
* fixup
* pad also returns the decoder attention mask, since that's useful to have
* clean up feature extractor logic
* pad can handle TTS task too
* remove stop_labels from loss calculation
* simplify logic
* fixup
* do -100 masking properly
* small STFT optimization (calculate mel filterbanks only once)
* replace torchaudio fbanks with audio_utils
* remove torchaudio dependency
* simplify & speed up the STFT
* don't serialize window and mel filters
* output cross attentions when generating speech
* add guided attention loss
* fix failing test
* Update src/transformers/models/speecht5/feature_extraction_speecht5.py
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
* Update src/transformers/models/speecht5/modeling_speecht5.py
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
* change type annotation of attention_mask to LongTensor
* extract loss into class
* remove unused frame_signal_scale argument
* use config object in loss class
* fix type annotations in doc comments
* change optional to just bool
* implement missing tokenizer method
* add deprecation warning
* Update src/transformers/models/speecht5/feature_extraction_speecht5.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/models/speecht5/feature_extraction_speecht5.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* add deprecation warning for stop_labels
---------
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-04-18 10:12:30 +01:00
Sylvain Gugger
dacd34568d
Mark auto models as important ( #22815 )
...
* Mark auto models as important
* Annoying file with bad line endings
2023-04-17 15:33:01 -04:00
Zachary Mueller
03462875cc
Introduce PartialState as the device handler in the Trainer ( #22752 )
...
* Use accelerate for device management
* Add accelerate to setup
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-04-17 15:09:45 -04:00
Sylvain Gugger
50caa20628
Revert "Use code on the Hub from another repo" ( #22813 )
...
Revert "Use code on the Hub from another repo (#22698 )"
This reverts commit ea7b0a539a
2023-04-17 14:22:13 -04:00
Sylvain Gugger
e13d6ef7dc
Simplify update metadata job ( #22811 )
...
* Simplify update metadata job
* Match more branch names
* Install all what is necessary
* Install all what is necessary
* Forgot the dev
* Install less stuff
* This syntax?
2023-04-17 13:54:20 -04:00
Zachary Mueller
cd3e0211a6
Remove accelerate from tf test reqs ( #22777 )
...
Remove accelerate from tf
2023-04-17 12:31:21 -04:00
Kunhao ZHENG
f8c43c9425
Fix squeeze into torch 1.x compatible form in llama model ( #22808 )
...
fix-squeeze-tuple
2023-04-17 17:28:48 +01:00
Yih-Dar
5269718cb7
Don't use LayoutLMv2 and LayoutLMv3 in some pipeline tests ( #22774 )
...
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-04-17 17:45:20 +02:00
Sylvain Gugger
ea7b0a539a
Use code on the Hub from another repo ( #22698 )
...
* initial work
* Add other classes
* Refactor code
* Move warning and fix dynamic pipeline
* Issue warning when necessary
* Add test
2023-04-17 11:36:29 -04:00
Wonhyeong Seo
4d2c52e830
🌐 [i18n-KO] Translated tasks/translation.mdx to Korean ( #22805 )
...
docs: ko: tasks/translation.mdx
2023-04-17 11:30:17 -04:00
Matt
2237127a6c
Fix sneaky torch dependency in TF example ( #22804 )
2023-04-17 16:11:52 +01:00
fpgaminer
626c1b8af1
improve(llama): Faster apply_rotary_pos_emb ( #22785 )
2023-04-17 15:18:38 +01:00
