Sayak Paul
4116d1ec75
[Examples/TensorFlow] minor refactoring to allow compatible datasets to work ( #22879 )
...
minor refactoring to allow compatible datasets to work.
2023-04-20 18:21:01 +05:30
Younes Belkada
10dd3a7d1c
[SAM] Change to facebook/sam-vit-base ( #22891 )
...
change to `facebook/sam-vit-base`
2023-04-20 14:11:18 +02:00
Quentin Ambard
aa43a76538
fix warning function call creating logger error (max_length and max_new_tokens) ( #22889 )
2023-04-20 13:08:03 +01:00
Yih-Dar
aa4316757d
Change schedule CI time ( #22884 )
...
* fix
* Update .github/workflows/self-nightly-past-ci-caller.yml
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
2023-04-20 14:01:08 +02:00
xloem
d50db469c0
Generation: only search for eos_token if set ( #22875 )
...
Generation: only check for eos_token if set
The check for unfinished_sequences.max(), which is to find sequences
that have ended early via eos_token_id, creates a synchronization point
even when there is no eos_token, which slows inference down.
This change moves the calculation to inside the condition checking for
eos_token, so that such slowdown may be removed by disabling this token.
Co-authored-by: John Doe <john.doe@example.com>
2023-04-20 12:18:28 +01:00
Oscar
a438a0941c
fix: Correct small typo in docstring ( #22857 )
...
* fix: Correct small typo in docstring
* fix: Run make fixup
2023-04-20 11:58:52 +01:00
fxmarty
4cfe328bae
Fix SAM example in documentation ( #22887 )
...
fix sam example
2023-04-20 12:22:42 +02:00
Shanmugam Ramasamy
cb47293eba
Patching clip model to create mask tensor on the device ( #22711 )
...
* Patching clip model to create mask tensor on the device
* Addressing PR's comments
* Addressing PR's comments
* Addressing PR's comments
---------
Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com>
2023-04-20 10:58:52 +01:00
Younes Belkada
2da73f6302
[SAM] Correct arxiv link ( #22886 )
...
put correct link
2023-04-20 11:23:12 +02:00
Joao Gante
4060d6857e
XGLM: Fix left-padding (PT and TF) ( #22828 )
2023-04-20 10:01:56 +01:00
Arthur
474bf508df
Add Segment Anything Model (SAM) ( #22654 )
...
* initial commit
* keys match
* update, fix conversion
* fixes, inference working
* fix
* more fixes
* more fixes
* clean up
* more clean up
* fix copies and add convext copied layer norm
* stash
* pretty big upfate
* cleaning
* more cleaning
* fixup stuffs
* fix copies
* fix iinit
* update test removing tokenizer
* nits
* add pretrained
* more nits
* remove tracking of pipeline
* few fixes
* update san and conversion script
* fix mask decoder and prompt encoder conversion
* fixes
* small update
* fix order
* fix
* fix image embeddings
* nites
* few fixes
* fix logits
* clean up
* fixes boxes inference
* v1 AMG
* clean up
* some clean up
* multi points support
* amg working
* fixup
* clean up
* readme
* update toctree
* fix type hint
* multiple fixes
* fixup
* fixes
* updates
* updates
* more tests
* few fixes
* change to `SamForMaskGeneration`
* doc
* fixup
* fix more tests
* multiple fixes
* fix CI tests
* refactor processor
* renamings
* draft the pipeline
* refactor
* fix tests
* fix test
* few cleanings
* fix test
* edit pipelien support chunking
* udate
* add slow tests
* fix nit
* fixup
* fix nit
* current chunk pipleine
* cast boxes in fp32
* nit
* current updates
* piepleine works
* fixup
* clean up config
* fix slow tests
* fix slow tests
* clean up
* update doc and pipeline
* adds more slow tests
* fix slow tests
* cleaning
* tests pass
* add docstring
* fix copies
* clean up
* support batch of images
* style
* dummy is needed, add tests
* fix slow tests
* fix CI
* update
* adds more tests
* fixes
* fixes
* fixup
* fixes
* few fixes
* filter
* few fixes
* some refactor
* touches finales
* fix
* style
* remove pipeline files
* fixes nits
* revert pipeline changes
* fix test
* fixup
* remove automodel for automatic mask generation
* fix failing torch tests
* update mdx
* revert removal of `MODEL_FOR_AUTOMATIC_MASK_GENERATION_MAPPING`
* update sam config based on review
Co-authored-by: amyeroberts <aeroberts4444@gmail.com>
Co-authored-by: sgugger <sylvain.gugger@gmail.com>
* update low_resolution_masks -> pred_masks
inti ln with layer_norm_eps
add_decomposed_rel_pos doc
forward doc of SamForMaskGeneration
* update processor docstring
* remove image processor import empty
* update for testing
* output vision hidden states + clean recomm
also test all iou values
* fixup
* fixup
* remove unused
* Update src/transformers/models/sam/modeling_sam.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/models/sam/image_processing_sam.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* nits
* fix
* fix CI tests and slow tests
* replace with Amy's processor
* clearer docstring
* add `SamVisionNeck`
* refactor - all CI tests should pass
* fix broken import on Gcolab
* few fixes here and there
* fix another bug
* fix more bugs
* update and merge
* correct ckpt
* address comments
* add tips
* revert
* fix docstring
* replace with `SamModel`
* make fixup
* add support for bathed images and batch ed points
* make fixup this time, really
* make fixup again and again
* few fixes here and there, this should be the touche finale
* Update docs/source/en/model_doc/sam.mdx
* fixup
* correct checkpoints
* correct name
* rm unneeded file
* add notebook
---------
Co-authored-by: younesbelkada <younesbelkada@gmail.com>
Co-authored-by: amyeroberts <aeroberts4444@gmail.com>
Co-authored-by: sgugger <sylvain.gugger@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
2023-04-19 21:01:49 +02:00
Matt
898efca72a
Fix to removing ESM special tokens ( #22870 )
...
Fix to make sure the EOS token doesn't come back
2023-04-19 19:42:29 +01:00
Zachary Mueller
a8aad0ec93
Fixup multigpu local_rank ( #22869 )
...
Fixup multigpu tests
2023-04-19 14:37:16 -04:00
Yih-Dar
06bab00338
Remove some pipeline skip cases ( #22865 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-04-19 20:27:19 +02:00
Yih-Dar
648bd5a8aa
Show diff between 2 CI runs on Slack reports ( #22798 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-04-19 19:27:37 +02:00
amyeroberts
5f97bbc124
Remove 'main' from doc links ( #22860 )
2023-04-19 15:03:57 +01:00
Yih-Dar
4603fe9b1f
use accelerate@main in CI ( #22859 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-04-19 14:58:53 +02:00
Elabonga Atuo
337225ec1c
feat(model parallelism): move labels to the same device as logits for M2M100 ( #22850 )
...
moved logits for m2m_100
2023-04-19 08:54:27 -04:00
Liu Chenyang
6bd8ae2640
move preprocess_logits_for_metrics before _nested_gather in trainer.e… ( #22603 )
...
* move preprocess_logits_for_metrics before _nested_gather in trainer.evaluation_loop
* fix
* Update src/transformers/trainer.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* fix
* fix
---------
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-04-19 08:53:47 -04:00
Matthijs Hollemans
c582e8aad0
fix SpeechT5 doc comments ( #22854 )
...
fix doc comments
2023-04-19 14:10:40 +02:00
Youssef Adarrab
84a6570e7b
Make ClipSeg compatible with model parallelism ( #22844 )
2023-04-18 19:31:59 -04:00
Zachary Mueller
5bb4ec6233
Raise err if minimum Accelerate version isn't available ( #22841 )
...
* Add warning about accelerate
* Version block Accelerate
* Include parse
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Check partial state
* Update param
---------
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-04-18 14:25:02 -04:00
Sylvain Gugger
5f09219400
Fix from_pretrained when model is instantiated on the meta device ( #22837 )
2023-04-18 13:54:18 -04:00
Sylvain Gugger
5f9b825c89
Use code on the Hub from another repo ( #22814 )
...
* initial work
* Add other classes
* Refactor code
* Move warning and fix dynamic pipeline
* Issue warning when necessary
* Add test
* Do not skip auto tests
* Fix failing tests
* Refactor and address review comments
* Address review comments
2023-04-18 13:46:11 -04:00
Zachary Mueller
aec10d162f
Update accelerate version + warning check fix ( #22833 )
2023-04-18 12:51:32 -04:00
Joao Gante
78cda46f17
Generate: Add assisted generation ( #22211 )
...
* working mvp
* remove breakpoint
* fix commit
* standardize outputs
* tmp commit
* tests almost ready
* tmp commit
* skip a few models
* Add streaming; Docs and examples
* document limitations
* PR commits
* Amy PR comments
2023-04-18 17:36:56 +01:00
Yih-Dar
90247d3e01
Fix test_eos_token_id_int_and_list_top_k_top_sampling ( #22826 )
...
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-04-18 16:04:51 +02:00
Yih-Dar
1ebc1dee92
Fix Past CI not running against the latest main ( #22823 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-04-18 15:41:41 +02:00
Gabriel Yang
42288269c3
🌐 [i18n-KO] Fix anchor links for docs auto_tutorial, training ( #22796 )
...
docs: ko: fix anchor links for docs (auto_tutorial, training)
Co-authored-by: Hyeonseo Yun <0525_hhgus@naver.com>
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-authored-by: Na Yeon Han <nayeon2.han@gmail.com>
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>
2023-04-18 09:11:30 -04:00
Matthijs Hollemans
ac2bc50a10
TTS fine-tuning for SpeechT5 ( #21824 )
...
* wrong argument name
* append eos_token_id
* all tokenizers need mask and ctc_blank tokens
* remove reduction factor from feature extractor
* add proper TTS loss
* did shifting the wrong way around
* mask out padded portions
* remove logits again (don't really need it)
* fix unit tests
* fixup
* pad also returns the decoder attention mask, since that's useful to have
* clean up feature extractor logic
* pad can handle TTS task too
* remove stop_labels from loss calculation
* simplify logic
* fixup
* do -100 masking properly
* small STFT optimization (calculate mel filterbanks only once)
* replace torchaudio fbanks with audio_utils
* remove torchaudio dependency
* simplify & speed up the STFT
* don't serialize window and mel filters
* output cross attentions when generating speech
* add guided attention loss
* fix failing test
* Update src/transformers/models/speecht5/feature_extraction_speecht5.py
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
* Update src/transformers/models/speecht5/modeling_speecht5.py
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
* change type annotation of attention_mask to LongTensor
* extract loss into class
* remove unused frame_signal_scale argument
* use config object in loss class
* fix type annotations in doc comments
* change optional to just bool
* implement missing tokenizer method
* add deprecation warning
* Update src/transformers/models/speecht5/feature_extraction_speecht5.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/models/speecht5/feature_extraction_speecht5.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* add deprecation warning for stop_labels
---------
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-04-18 10:12:30 +01:00
Sylvain Gugger
dacd34568d
Mark auto models as important ( #22815 )
...
* Mark auto models as important
* Annoying file with bad line endings
2023-04-17 15:33:01 -04:00
Zachary Mueller
03462875cc
Introduce PartialState as the device handler in the Trainer ( #22752 )
...
* Use accelerate for device management
* Add accelerate to setup
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-04-17 15:09:45 -04:00
Sylvain Gugger
50caa20628
Revert "Use code on the Hub from another repo" ( #22813 )
...
Revert "Use code on the Hub from another repo (#22698 )"
This reverts commit ea7b0a539a
2023-04-17 14:22:13 -04:00
Sylvain Gugger
e13d6ef7dc
Simplify update metadata job ( #22811 )
...
* Simplify update metadata job
* Match more branch names
* Install all what is necessary
* Install all what is necessary
* Forgot the dev
* Install less stuff
* This syntax?
2023-04-17 13:54:20 -04:00
Zachary Mueller
cd3e0211a6
Remove accelerate from tf test reqs ( #22777 )
...
Remove accelerate from tf
2023-04-17 12:31:21 -04:00
Kunhao ZHENG
f8c43c9425
Fix squeeze into torch 1.x compatible form in llama model ( #22808 )
...
fix-squeeze-tuple
2023-04-17 17:28:48 +01:00
Yih-Dar
5269718cb7
Don't use LayoutLMv2 and LayoutLMv3 in some pipeline tests ( #22774 )
...
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-04-17 17:45:20 +02:00
Sylvain Gugger
ea7b0a539a
Use code on the Hub from another repo ( #22698 )
...
* initial work
* Add other classes
* Refactor code
* Move warning and fix dynamic pipeline
* Issue warning when necessary
* Add test
2023-04-17 11:36:29 -04:00
Wonhyeong Seo
4d2c52e830
🌐 [i18n-KO] Translated tasks/translation.mdx to Korean ( #22805 )
...
docs: ko: tasks/translation.mdx
2023-04-17 11:30:17 -04:00
Matt
2237127a6c
Fix sneaky torch dependency in TF example ( #22804 )
2023-04-17 16:11:52 +01:00
fpgaminer
626c1b8af1
improve(llama): Faster apply_rotary_pos_emb ( #22785 )
2023-04-17 15:18:38 +01:00
Jungnerd
abbc96a214
[i18n-KO] fix: docs: ko: sagemaker anchors and _toctree.yml ( #22549 )
...
fix: docs: ko: sagemaker anchors and `_toctree.yml`
Co-authored-by: Hyeonseo Yun <0525_hhgus@naver.com>
Co-authored-by: Gabriel Yang <gabrielwithhappy@gmail.com>
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-authored-by: Na Yeon Han <nayeon2.han@gmail.com>
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
2023-04-17 07:41:52 -04:00
Na Yeon Han
18c894814e
🌐 [i18n-KO] Translated custom_models.mdx to Korean ( #22534 )
...
docs: ko: translated `custom_models.mdx`
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
Co-authored-by: Gabriel Yang <gabrielwithhappy@gmail.com>
Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>
2023-04-17 07:39:53 -04:00
Yih-Dar
76d24f1a83
Fix test_word_time_stamp_integration for Wav2Vec2ProcessorWithLMTest ( #22800 )
...
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-04-17 12:41:55 +02:00
bcol
28f26c107b
Generate: add CJK support to TextStreamer ( #22664 )
2023-04-15 10:35:08 +01:00
oscar-garzon
fb3aa06cb6
Move labels to the same device as logits for Whisper ( #22779 )
2023-04-14 19:08:41 -04:00
amyeroberts
20e54e49fa
Indexing fix - CLIP checkpoint conversion ( #22776 )
...
* Indexing fix - CLIP checkpoint conversion
* Fix up
2023-04-14 19:12:47 +01:00
Joao Gante
895ae3b5c4
Seq2SeqTrainer: Evict decoder_input_ids only when it is created from labels ( #22772 )
2023-04-14 17:45:14 +01:00
Mayank Agarwal
daf53241d6
Fix word_ids hyperlink ( #22765 )
...
* Fix word_ids hyperlink
* Add suggested fix
2023-04-14 16:18:15 +01:00
Matt
06e737fbaf
Tweak ESM tokenizer for Nucleotide Transformer ( #22770 )
...
* If EOS is None, don't add it to sequences
* If EOS is None, don't add it to sequences
2023-04-14 15:18:43 +01:00
