Yoni Gozlan
1456120929
Uniformize kwargs for Udop processor and update docs ( #33628 )
...
* Add optional kwargs and uniformize udop
* cleanup Unpack
* nit Udop
2024-09-23 12:47:32 -04:00
Isotr0py
be9cf070ee
Fix Llava conversion for LlavaQwen2ForCausalLM with Clip vision tower ( #33613 )
...
fix llavaqwen2 model conversion
2024-09-23 12:07:15 +01:00
chengchengpei
214db9e660
add back self.max_position_embeddings = config.max_position_embeddings ( #33550 )
...
* add back self.max_position_embeddings = config.max_position_embeddings
* fix-copies
2024-09-23 12:54:58 +02:00
Pablo Montalvo
6d02968d51
handle dependency errors in check_imports ( #33622 )
...
* handle dependency errors in check_imports
* change log level to warning
2024-09-23 12:38:52 +02:00
Pablo Montalvo
b7c381f011
Fix DPT /Dinov2 sdpa regression on main ( #33660 )
...
* fallback to eager if output attentions.
* fix copies
2024-09-23 11:49:16 +02:00
Pablo Montalvo
9eb93854b9
Clean up Unpack imports ( #33631 )
...
clean up Unpack imports
2024-09-23 10:21:17 +02:00
Avishai Elmakies
78b2929c05
Sdpa dino v2 ( #33403 )
...
* add sdpa to dinov2
* fixup
* add dinov2 to sdpa doc
* update doc order
* [run-slow] dinov2
* common to eager
* [run-slow] dinov2
* update attn implementation in common
* update test_modeling_dinov2 to have mask_ration, num_masks and mask_length similar to vit
* [run-slow] dinov2
---------
Co-authored-by: Avishai Elmakies <avishai.elma@cs.huji.ac.il>
2024-09-21 01:58:00 +01:00
amyeroberts
e71bf70e33
Pixtral update example checkpoint ( #33633 )
...
* Update pixtral example checkpoint
* Fix typo
2024-09-21 01:01:16 +01:00
Mayank Mishra
e472e077c2
Granitemoe ( #33207 )
...
* first commit
* drop tokenizer
* drop tokenizer
* drop tokenizer
* drop convert
* granite
* drop tokenization test
* mup
* fix
* reformat
* reformat
* reformat
* fix docs
* stop checking for checkpoint
* update support
* attention multiplier
* update model
* tiny drop
* saibo drop
* skip test
* fix test
* fix test
* drop
* drop useless imports
* update docs
* drop flash function
* copied from
* drop pretraining tp
* drop pretraining tp
* drop pretraining tp
* drop unused import
* drop code path
* change name
* softmax scale
* head dim
* drop legacy cache
* rename params
* cleanup
* fix copies
* comments
* add back legacy cache
* multipliers
* multipliers
* multipliers
* text fix
* fix copies
* merge
* multipliers
* attention multiplier
* drop unused imports
* add granitemoe
* add decoration
* remove moe from sequenceclassification
* fix test
* fix
* fix
* fix
* move rope?
* merge
* drop bias
* drop bias
* Update src/transformers/models/granite/configuration_granite.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* fix
* Update src/transformers/models/granite/modeling_granite.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* fix
* fix
* fix
* fix
* drop
* drop
* fix
* fix
* cleanup
* cleanup
* fix
* fix granite tests
* fp32 test
* fix
* drop jitter
* fix
* rename
* rename
* fix config
* add gen test
---------
Co-authored-by: Yikang Shen <yikang.shn@gmail.com>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
2024-09-21 01:43:50 +02:00
jiqing-feng
49a0bef4c1
enable low-precision pipeline ( #31625 )
...
* enable low-precision pipeline
* fix parameter for ASR
* reformat
* fix asr bug
* fix bug for zero-shot
* add dtype check
* rm useless comments
* add np.float16 check
* Update src/transformers/pipelines/image_classification.py
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com>
* Update src/transformers/pipelines/token_classification.py
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com>
* fix comments
* fix asr check
* make fixup
* No more need for is_torch_available()
---------
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com>
Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>
Co-authored-by: Matt <rocketknight1@gmail.com>
2024-09-20 16:43:30 -07:00
litianjian
7b2b536a81
Fix typos ( #33583 )
...
Co-authored-by: litianjian <litianjian@bytedance.com>
2024-09-20 16:34:42 -07:00
GeLee
e9356a4206
Fix qwen2vl float16 inference bug ( #33312 )
...
* fix qwen2vl float16 inference bug
* [run-slow] qwen2_vl
2024-09-20 16:28:46 -07:00
Yih-Dar
75c878da1e
Update daily ci to use new cluster ( #33627 )
...
* update
* re-enable daily CI
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2024-09-20 21:05:30 +02:00
Yih-Dar
077b552f07
Fix some missing tests in circleci ( #33559 )
...
* fix
* fix
* fix
* fix
* skip
* skip more
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2024-09-20 20:58:51 +02:00
Joao Gante
77c5d59e0e
Generate: assistant should sample when the main model samples ( #33534 )
2024-09-20 17:01:49 +01:00
Duc-Viet Hoang
dc8b6eaeee
Fix contrastive search to correctly handle input with padding ( #33507 )
...
* fix: handle padding in contrastive search for decoder-only models
* fix: handle padding in contrastive search for encoder-decoder models
* tests: move padding contrastive test to test_util, add t5 test
* fix: handle if model_kwargs["decoder_attention_mask"] is None
* refactor: improve padding input contrastive search generation tests
* chore: _ranking_fast to use LongTensor for cosine_matrix_mask
2024-09-20 16:52:08 +01:00
Yoni Gozlan
c0c6815dc9
Add support for args to ProcessorMixin for backward compatibility ( #33479 )
...
* add check and prepare args for BC to ProcessorMixin, improve ProcessorTesterMixin
* change size and crop_size in processor kwargs tests to do_rescale and rescale_factor
* remove unnecessary llava processor kwargs test overwrite
* nit
* change data_arg_name to input_name
* Remove unnecessary test override
* Remove unnecessary tests Paligemma
* Move test_prepare_and_validate_optional_call_args to TesterMixin, add docstring
2024-09-20 11:40:59 -04:00
Yih-Dar
31caf0b95f
Fix missing test in torch_job ( #33593 )
...
fix missing tests
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2024-09-20 17:16:44 +02:00
Joao Gante
2fdb5e74cc
VLM generate: tests can't generate image/video tokens ( #33623 )
2024-09-20 15:43:27 +01:00
Omar Salman
653eb40425
Add sdpa for BioGpt ( #33592 )
...
* Add sdpa for BioGpt
* Updates
* Add the docs
* [run_slow] biogpt
* Use the copy mechanism to ensure consistency
* [run_slow] biogpt
2024-09-20 14:27:32 +01:00
amyeroberts
f9b4409726
Remove unnecessary CPM model tests ( #33621 )
...
Remove model tests
2024-09-20 14:20:57 +01:00
Joao Gante
266d0a6375
Generate: remove flakyness in test_generate_from_inputs_embeds_decoder_only ( #33602 )
...
almost zero is not zero
2024-09-20 14:50:42 +02:00
Lake Lee
ec1424c6a3
Update modeling_mamba2.py, fix pad size ( #32599 )
...
* Update modeling_mamba2.py
Fix pad_size calculation to ensure it's less than self.chunk_size
* [run_slow] mamba2
* [run-slow] mamba2
* [run-slow] Add @require_read_token decorator to failing tests for token propagation
* [run_slow] mamba2
2024-09-20 11:40:57 +01:00
Fanli Lin
8bd1f2f338
[tests] make more tests device-agnostic ( #33580 )
...
* enable
* fix
* add xpu skip
* add marker
* skip for xpu
* add more
* enable on accelerator
* add more cases
* add more tests
* add more
2024-09-20 10:16:43 +01:00
Yih-Dar
31650a53a1
Allow CI could be run on private forked repositories (e.g. new model additions) ( #33594 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2024-09-20 11:00:34 +02:00
Yih-Dar
6dc364616d
Fix CircleCI nightly run ( #33558 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2024-09-20 10:57:21 +02:00
Joao Gante
bdf4649f67
Docs: add the ability to manually trigger jobs ( #33598 )
2024-09-20 09:37:39 +01:00
Pedro Cuenca
0c718f16d1
Fix Llama 3 TikToken conversion ( #33538 )
...
* Fix Llama 3 TikToken conversion
* No need to add tokens again
2024-09-20 01:28:33 +02:00
Fanli Lin
4d8908df27
[tests] enable GemmaIntegrationTest on XPU ( #33555 )
...
enable GemmaIntegrationTest
2024-09-19 19:39:19 +01:00
Fanli Lin
b87755aa6d
[tests] skip tests for xpu ( #33553 )
...
* enable
* fix
* add xpu skip
* add marker
* skip for xpu
* add more
* add one more
2024-09-19 19:28:04 +01:00
Yoni Gozlan
f111d5b783
Uniformize kwargs for Paligemma processor and update docs ( #33571 )
...
* Uniformize paligemma processor
* nit
2024-09-19 14:14:06 -04:00
Joao Gante
52920b5dd5
Cache: don't throw warnings on gemma2 when instantiating a new cache ( #33595 )
2024-09-19 17:42:47 +01:00
Anton Vlasjuk
b50ff5993a
[Mamba2] Move dt calculations to kernel ( #33520 )
...
* use kernel for dt calculations
* add small test
* [run-slow] mamba2
2024-09-19 17:41:17 +01:00
Vladislav Bronzov
162056a3f4
change sequence_bias type of SequenceBiasLogitsProcessor to list, add… ( #33375 )
...
* change sequence_bias type of SequenceBiasLogitsProcessor tp list, add config tests for all processors
* fix format
* small fix for all_token_bias_pairs_are_valid internal func
* small typo fix in description
* improve test impl, some SequenceBiasLogitsProcessor refactoring
2024-09-19 17:35:44 +01:00
Joao Gante
d9d59e7bac
Generate: check that attention_mask is 2D ( #33575 )
...
check attention mask in generate
2024-09-19 16:23:17 +01:00
Pablo Montalvo
413008c580
add uniform processors for altclip + chinese_clip ( #31198 )
...
* add initial design for uniform processors + align model
* add uniform processors for altclip + chinese_clip
* fix mutable default 👀
* add configuration test
* handle structured kwargs w defaults + add test
* protect torch-specific test
* fix style
* fix
* rebase
* update processor to generic kwargs + test
* fix style
* add sensible kwargs merge
* update test
* fix assertEqual
* move kwargs merging to processing common
* rework kwargs for type hinting
* just get Unpack from extensions
* run-slow[align]
* handle kwargs passed as nested dict
* add from_pretrained test for nested kwargs handling
* [run-slow]align
* update documentation + imports
* update audio inputs
* protect audio types, silly
* try removing imports
* make things simpler
* simplerer
* move out kwargs test to common mixin
* [run-slow]align
* skip tests for old processors
* [run-slow]align, clip
* !$#@!! protect imports, darn it
* [run-slow]align, clip
* [run-slow]align, clip
* update common processor testing
* add altclip
* add chinese_clip
* add pad_size
* [run-slow]align, clip, chinese_clip, altclip
* remove duplicated tests
* fix
* update doc
* improve documentation for default values
* add model_max_length testing
This parameter depends on tokenizers received.
* Raise if kwargs are specified in two places
* fix
* match defaults
* force padding
* fix tokenizer test
* clean defaults
* move tests to common
* remove try/catch block
* deprecate kwarg
* format
* add copyright + remove unused method
* [run-slow]altclip, chinese_clip
* clean imports
* fix version
* clean up deprecation
* fix style
* add corner case test on kwarg overlap
* resume processing - add Unpack as importable
* add tmpdirname
* fix altclip
* fix up
* add back crop_size to specific tests
* generalize tests to possible video_processor
* add back crop_size arg
* fixup overlapping kwargs test for qformer_tokenizer
* remove copied from
* fixup chinese_clip tests values
* fixup tests - qformer tokenizers
* [run-slow] altclip, chinese_clip
* remove prepare_image_inputs
2024-09-19 17:21:54 +02:00
Pablo Montalvo
4f0246e535
fix tests with main revision and read token ( #33560 )
...
* fix tests with main revision and read token
* [run-slow]mamba2
* test previously skipped tests
* [run-slow]mamba2
* skip some tests
* [run-slow]mamba2
* finalize tests
* [run-slow]mamba2
2024-09-19 17:10:22 +02:00
Joao Gante
80b774eb29
Cache: don't show warning in forward passes when past_key_values is None ( #33541 )
2024-09-19 12:02:46 +01:00
Joao Gante
f3b3810fe6
rag: fix CI ( #33578 )
2024-09-19 11:55:26 +01:00
Raushan Turganbay
d7975a5874
VLMs: enable generation tests ( #33533 )
...
* add tests
* fix whisper
* update
* nit
* add qwen2-vl
* more updates!
* better this way
* fix this one
* fix more tests
* fix final tests, hope so
* fix led
* Update tests/generation/test_utils.py
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
* pr comments
* not pass pixels and extra for low-mem tests, very flaky because of visio tower
---------
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
2024-09-19 12:04:24 +02:00
Raushan Turganbay
e40bb4845e
Load and save video-processor from separate folder ( #33562 )
...
* load and save from video-processor folder
* Update src/transformers/models/llava_onevision/processing_llava_onevision.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
---------
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
2024-09-19 09:56:52 +02:00
Yoach Lacombe
5af7d41e49
Codec integration ( #33565 )
...
* clean mimi commit
* some nits suggestions from Arthur
* make fixup
* rename repo id + change readme
* Update docs/source/en/model_doc/mimi.md
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* add flaky flag to batching equivalence due to audio_codes failing sometimes
---------
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
2024-09-18 19:23:44 +02:00
Marc Sun
6019f3ff78
Fix bnb dequantization ( #33546 )
2024-09-18 19:10:28 +02:00
Yoni Gozlan
7b1ce634cb
Improve compiled RT-DETR inference speed ( #33412 )
...
* modify rt detr to improve inference times when compiled
* Remove redundant "to"
* Fix conditional lru_cache and missing shapes_list
* nit unnecessary list creation
* Fix compile error when ninja not available and custon kernel activated
2024-09-18 12:56:45 -04:00
Dominik Niedziela
9db963aeed
enforce original size to be a list ( #33564 )
...
* enforce original size to be a list
* formatting
* apply datatype change to unpad_image in llava_next
2024-09-18 16:38:31 +01:00
Matt
8efc06ee18
Return attention mask in ASR pipeline to avoid warnings ( #33509 )
...
return attention mask in ASR pipeline
2024-09-18 15:57:39 +01:00
Joao Gante
7542fac2c7
Pipeline: no side-effects on model.config and model.generation_config 🔫 ( #33480 )
2024-09-18 15:43:06 +01:00
Umar Butler
fc83a4d459
Added support for bfloat16 to zero-shot classification pipeline ( #33554 )
...
* Added support for bfloat16 to zero-shot classification pipeline
* Ensure support for TF.
Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>
* Remove dependency on `torch`.
Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>
---------
Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>
2024-09-18 15:41:50 +01:00
Yoach Lacombe
f883827c0a
Fix tests in ASR pipeline ( #33545 )
2024-09-18 16:25:45 +02:00
Ziyú Ye
4f1e9bae4e
fix the wandb logging issue ( #33464 )
...
* fix the wandb logging issue
* handle ConfigError in WandbCallback; move import to local scope
* update integration_utils.py; move import of ConfigError
* Update integration_utils.py: remove trailing whitespace
2024-09-18 07:23:05 -07:00
