co63oc
996f512d52
Fix typos in tests ( #36547 )
...
Signed-off-by: co63oc <co63oc@users.noreply.github.com>
2025-03-05 15:04:06 -08:00
ivarflakstad
c0c5acff07
Fix bamba tests amd ( #36535 )
2025-03-04 15:24:27 +01:00
Arthur
84f0186e89
Add aya ( #36521 )
...
* initial commit
* small fix
* move stuff to image processing file
* remove stuff in validate turn and fix return tensor
* remove liquid stuff
* in the process of addressing comments
* changes to get the right tokenization
* new __init__ works
* fixing defulat std and mean
* works
* small testing scipt -- to be deleted before merge
* remove redundant code
* addressing comments
* fix inits, add docs templates
* refactor processor, switch to gotocr image processor
* remove image proc from init
* refactor to working llava-style architecture
* Change AyaVisionModel to AyaVisionForConditionalGeneration
* add tests
* fixups
* update doc
* Adding logits_to_keep explicitly in ayavision forward to enable compatibility with cohere model
* better variable names + remove code paths
* Updates to aya_vision.md
* address comments
* adding copied from
* make style and remove unused projector_hidden_act from config
* sort init
* include usage of fast image proc and proc on cuda in doc
* update checkpoint iin test processor
* update checkpoint in test processor 2
* remove test_model and update docstring
* skip failing tests
---------
Co-authored-by: Saurabh Dash <saurabh@cohere.com>
Co-authored-by: yonigozlan <yoni.gozlan@huggingface.co>
2025-03-04 12:24:33 +01:00
Steven Liu
c0f8d055ce
[docs] Redesign ( #31757 )
...
* toctree
* not-doctested.txt
* collapse sections
* feedback
* update
* rewrite get started sections
* fixes
* fix
* loading models
* fix
* customize models
* share
* fix link
* contribute part 1
* contribute pt 2
* fix toctree
* tokenization pt 1
* Add new model (#32615 )
* v1 - working version
* fix
* fix
* fix
* fix
* rename to correct name
* fix title
* fixup
* rename files
* fix
* add copied from on tests
* rename to `FalconMamba` everywhere and fix bugs
* fix quantization + accelerate
* fix copies
* add `torch.compile` support
* fix tests
* fix tests and add slow tests
* copies on config
* merge the latest changes
* fix tests
* add few lines about instruct
* Apply suggestions from code review
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* fix
* fix tests
---------
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* "to be not" -> "not to be" (#32636 )
* "to be not" -> "not to be"
* Update sam.md
* Update trainer.py
* Update modeling_utils.py
* Update test_modeling_utils.py
* Update test_modeling_utils.py
* fix hfoption tag
* tokenization pt. 2
* image processor
* fix toctree
* backbones
* feature extractor
* fix file name
* processor
* update not-doctested
* update
* make style
* fix toctree
* revision
* make fixup
* fix toctree
* fix
* make style
* fix hfoption tag
* pipeline
* pipeline gradio
* pipeline web server
* add pipeline
* fix toctree
* not-doctested
* prompting
* llm optims
* fix toctree
* fixes
* cache
* text generation
* fix
* chat pipeline
* chat stuff
* xla
* torch.compile
* cpu inference
* toctree
* gpu inference
* agents and tools
* gguf/tiktoken
* finetune
* toctree
* trainer
* trainer pt 2
* optims
* optimizers
* accelerate
* parallelism
* fsdp
* update
* distributed cpu
* hardware training
* gpu training
* gpu training 2
* peft
* distrib debug
* deepspeed 1
* deepspeed 2
* chat toctree
* quant pt 1
* quant pt 2
* fix toctree
* fix
* fix
* quant pt 3
* quant pt 4
* serialization
* torchscript
* scripts
* tpu
* review
* model addition timeline
* modular
* more reviews
* reviews
* fix toctree
* reviews reviews
* continue reviews
* more reviews
* modular transformers
* more review
* zamba2
* fix
* all frameworks
* pytorch
* supported model frameworks
* flashattention
* rm check_table
* not-doctested.txt
* rm check_support_list.py
* feedback
* updates/feedback
* review
* feedback
* fix
* update
* feedback
* updates
* update
---------
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>
2025-03-03 10:33:46 -08:00
Matt
1975be4d97
Fix edge case for continue_final_message ( #36404 )
...
* Fix edge case for continue_final_message
* lstrip() correctly
* Add regression test
* Add a clearer error message when the final message is not present
* Add a clearer error message when the final message is not present
* Fix massive bug!
2025-03-03 18:03:03 +00:00
Matt
2aff938992
Fix pipeline+peft interaction ( #36480 )
...
* Fix pipeline-peft interaction
* once again you have committed a debug breakpoint
* Remove extra testing line
* Add a test to check adapter loading
* Correct adapter path
* make fixup
* Remove unnecessary check
* Make check a little more stringent
2025-03-03 18:01:43 +00:00
Zach Mueller
4d8259d245
Fix loading zero3 weights ( #36455 )
...
* Check if fixes
* Fix zero3 loading
* Quality
* Fix marc nit
* Add fast tests
* Migrate to integrations.deepspeed rather than modeling_utils
* Style
2025-03-03 15:05:58 +01:00
Yoni Gozlan
2c5d038f92
Add Got-OCR 2 Fast image processor and refactor slow one ( #36185 )
...
* refactor image processor slow got ocr
* add working image processor fast
* fix fast image processor, update doc
* use one big loop for processing patches
2025-03-01 00:56:00 -05:00
Eduardo Pacheco
222505c7e4
[GroundingDino] Fix grounding dino loss 🚨 ( #31828 )
...
* Starting to fix GroundingDinoLoss and GroundingDinoHungarianMatcher
* More updates
* More updates
* fixed: GroundingDinoLoss
* fixed: failing tests
* Update src/transformers/models/grounding_dino/modeling_grounding_dino.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update tests/models/grounding_dino/test_modeling_grounding_dino.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/grounding_dino/modeling_grounding_dino.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/grounding_dino/modeling_grounding_dino.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/grounding_dino/modeling_grounding_dino.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/grounding_dino/modeling_grounding_dino.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/grounding_dino/modeling_grounding_dino.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/grounding_dino/modeling_grounding_dino.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/grounding_dino/modeling_grounding_dino.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/grounding_dino/modeling_grounding_dino.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Addressed comments
* Update src/transformers/models/grounding_dino/modeling_grounding_dino.py
Co-authored-by: Sangbum Daniel Choi <34004152+SangbumChoi@users.noreply.github.com>
* add: cardinality loss and make box loss as copy from
* change: default for reduction loss is sum
* fix: vectorized generate fake box
* fix copies
* Addressed comments
* addressed comments
* addressed one-hot
* Update tests/models/grounding_dino/test_modeling_grounding_dino.py
Co-authored-by: Sangbum Daniel Choi <34004152+SangbumChoi@users.noreply.github.com>
* Addressed comments
* fixed test
* Update src/transformers/models/grounding_dino/modeling_grounding_dino.py
* Update tests/models/grounding_dino/test_modeling_grounding_dino.py
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com>
* Starting to fix GroundingDinoLoss and GroundingDinoHungarianMatcher
* More updates
* More updates
* fixed: GroundingDinoLoss
* add: cardinality loss and make box loss as copy from
* fix copies
* Revert "Update tests/models/grounding_dino/test_modeling_grounding_dino.py"
This reverts commit aa74c4c57c430e54cc74c414d6269edb65c73e83.
* [run-slow] groundigdino
* remove nestedtensor
* [run-slow] groundig_dino
* [run-slow] grounding_dino
* [run-slow] grounding_dino
* [run-slow] grounding_dino
* check
* check
* add: enconder intermediate outputs to ImageLoss forward
* add: GroundingDinoForObjectDetectionLoss in the loss directory
* make style
* fix the loss function
* remove class_reduction since it sum is default
* remove class_reduction
* Update src/transformers/loss/loss_grounding_dino.py
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com>
* simple fix
* Update src/transformers/loss/loss_grounding_dino.py
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com>
* minor fix
* Update src/transformers/loss/loss_for_object_detection.py
---------
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
Co-authored-by: Sangbum Daniel Choi <34004152+SangbumChoi@users.noreply.github.com>
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com>
Co-authored-by: sangbumchoi <danielsejong55@gmail.com>
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2025-02-27 19:15:58 +00:00
Yih-Dar
482d17be60
Fix hub_retry ( #36449 )
...
* cry
* trigger
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2025-02-27 14:38:25 +01:00
Joao Gante
8aed019764
[generate] torch.distributed-compatible DynamicCache ( #36373 )
...
* test
* docstring
* prepare distributed cache data
* fix cat dim
* test mvp
* add test checks
* like this?
* working test and solution
* nit
* nit
* add shape info
2025-02-27 11:48:57 +00:00
Mohamed Mekkouri
a7fbab33ae
Fix Expected output for compressed-tensors tests ( #36425 )
...
fix
2025-02-26 21:17:24 +01:00
Arthur
1603018e7a
Update form pretrained to make TP a first class citizen ( #36335 )
...
* clean code
* oups
* fix merge
* yups
* fix if
* now you can play
* fix shape issue
* try non blocking
* fix
* updates
* up
* updates
* fix most of thetests
* update
* update
* small updates
* up
* fix the remaining bug?
* update
* rename when you read from the file
* buffer issues
* current status
* cleanup
* properly allocate dumb memory
* update a small bug
* fix colwise rep issue
* fix keep in float 32 that was keeping everything in float 32
* typo
* more fixes with keep_in_fp32_modules as we use to serach on it
* fix ROPE dtype for TP
* remove what's breaking the tests
* updates
* update and fixes
* small cleanup after merging
* allocate 2x to be safe
* style, auto
* update
* yup nit
* fix
* remove slow as fuck torch api :(
* work
* fixup
* update
* brting the fix back
* fix and update
* fixes
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com>
* updates because some suggestions were wrong 👀
* update?
* fuck this bloated function
* typo
* fix the dumb prefix thing once and forall
* fixes here and there
* updates
* remove prints
* fix strict cases
* styel
* properly fix keys on load!
* update
* fix base model prefix issue
* style
* update
* fix all?
* remoce 1 print
* fix the final etsts
* fixup
* last nits
* fix the detach issue which cause a 2x slowdown
* fixup
* small fixes
* ultra nit
* fix
* fix
---------
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com>
2025-02-26 20:12:38 +01:00
Nadav Timor
d18d9c3205
Universal Speculative Decoding CandidateGenerator ( #35029 )
...
* move `TestAssistedCandidateGeneratorDifferentTokenizers` into a new testing file
* refactor
* NOTHING. add space to rerun github actions tests
* remove it...
* `UniversalSpeculativeDecodingGenerator`
* Use `UniversalSpeculativeDecodingGenerator` when `generation_config.do_sample=True`
* assistant tokenizes only the target's new suffix
* formatting
* fix code
* fix code
* formatting
* add `TestGenerateWithDifferentModels`
* `TestGenerateWithDifferentModels` parameterize on `do_sample`
* `AssistantVocabMapping` & `AssistantVocabMappingCache`
* formatting
* `AssistantToTargetTranslator`: `get_target_input_ids` & `get_target_logits`
* improve `_get_assistant_to_target_input_ids` & formatting
* renaming
* WIP: debugging `min_new_tokens`
* fix get_target_ids
* `UniversalSpeculativeDecodingGenerator`
* assistant tokenizes only the target's new suffix
* formatting
* fix code
* fix code
* formatting
* `TestGenerateWithDifferentModels` parameterize on `do_sample`
* `AssistantVocabMapping` & `AssistantVocabMappingCache`
* formatting
* `AssistantToTargetTranslator`: `get_target_input_ids` & `get_target_logits`
* improve `_get_assistant_to_target_input_ids` & formatting
* renaming
* WIP: debugging `min_new_tokens`
* fix get_target_ids
* fix device issue
* fix get_assistant_input_ids
* add `TestAssistedCandidateGeneratorDifferentTokenizers`
* formatting
* `AssistantVocabTranslatorCache` refactor & tests
* revert changes in `src/transformers/generation/logits_process.py`
* refactor `AssistedCandidateGenerator`
* refactor `AssistedCandidateGeneratorDifferentTokenizers`
* formatting
* refactor `UniversalSpeculativeDecodingGenerator`
* fix negative value for max_new_tokens
* fix generation length target + attention_mask vs. assistant + attent
* fix device
* fix negative max_new_tokens bug
* fix UAG
* minor
* formatting
* `AssistedCandidateGeneratorDifferentTokenizers` `lookbehind`s init
* resolve conflict & formatting
* rerun CI tests
* remove space...
* remove old code
* fix candidate_input_ids device
* minor
* formatting
* Fix prepare + apply (#7 )
* fix prepare + apply
* move to cpu
* simplity suppress_tokens
* fix bugs and refacatoring
* device move
* handle self.config.vocab_size > len(target_tokenizer.get_vocab())
* no need to normalize in candidate_generator
* address Nadav's comments + minor
* optimize device move + SuppressTokensLogitsProcessor
* AssistantToTargetTranslator, SuppressTokensLogitsProcessor and tokenizers mapping improvements
* padding size
* padding improvement
* fix and simplify get_target_logits
* renaming in get_target_logits
* minor
* add filter_value and suppress_tokens_id
* style + rename
* remove TODO
* restore original SelectTokensLogitsProcessor with modification
* fix style
* fix _update_past_and_masks and optimize code
* remove assistant_vocab_size arg
* fix attention_mask
* call _prepare_attention_mask also if not has_past_key_values
* handling attention mask for first generation
* comment
* restore test
* remove SelectTokensLogitsProcessor
* _update_past_and_masks implementation for USD
* Add unittests for Universal Assisted generation
* fix style
* update tests
* Remove unused import and fix `test_speculation_depth` test
* exclude special and reserved tokens from tokenizer for UAG
* mv `test_universal_assisted_generation.py` to `generation/test_candidate_generator.py`
* Remove unused imports and fix style using `make style` (#9 )
* formatting
* Swap gated `meta-llama/llama-3.2` with `allenai/llama` (#10 )
* Fix space sign disagreement (#12 )
* default values for AssistantToTargetTranslator fileds
* fix space sign
* minor
* fix test + style
* Default values for some fields of assistant to target translator (#11 )
* default values for AssistantToTargetTranslator fileds
* fix
* add support to empty logit_processors
* Update candidate_generator.py (#15 )
fix typo
* BUG fix in _prepare_assistant_input_ids (#14 )
* fix _prepare_assistant_input_ids
* target_to_assistant_input_ids
* Update src/transformers/generation/candidate_generator.py
Co-authored-by: Nadav Timor <nadav.timor@weizmann.ac.il>
---------
Co-authored-by: Nadav Timor <nadav.timor@weizmann.ac.il>
* typo (`target_to_assistant_input_ids`)
* formatting
* merge upstream/main
* Fix minor review comments (#16 )
* Fix: `token_ids.to(torch.int64)` (#18 )
* tok ids to `torch.int64` (reference: https://huggingface.co/docs/transformers.js/en/api/tokenizers )
* `LongTensor`
* fix dtype
* `assistant_input_ids.to(dtype=torch.long)`
* Remove unused import from test_candidate_generator.py
* Remove unused import from test_candidate_generator.py
* Remove `numpy` import
* resolve pr comments (#19 )
* `AssistantToTargetTranslator` docstring
* (per gante's comment) `filter_value` and `suppress_tokens_id` to class constants
* update `AssistantToTargetTranslator` docstring
* (gante's comment) replace `match-case`
* formatting
* Fix Joao's comments (#21 )
* remove threading
* fix logits_processor
* fix test device
* fix style (#23 )
* Move atm (#24 )
* move AssistantToTargetTranslator
* fixup
* fix logit_processor
* add atm_translator test
* refactor test
* remove threading from test
* add require_torch in tests
* move AssistantVocabTranslatorCache + add tests
* ruff fix
---------
Co-authored-by: jmamou <jonathan.mamou@intel.com>
Co-authored-by: Gaurav <gauravj@d-matrix.ai>
Co-authored-by: Gaurav Jain <gaurjain14@gmail.com>
Co-authored-by: gauravjain14 <41287729+gauravjain14@users.noreply.github.com>
2025-02-26 16:14:02 +00:00
Manny Cortes
082834dd79
fix: prevent model access error during Optuna hyperparameter tuning ( #36395 )
...
* fix: prevent model access error during Optuna hyperparameter tuning
The `transformers.integrations.integration_utils.run_hp_search_optuna` function releases model memory and sets trainer.model to None after each trial. This causes an AttributeError when subsequent Trainer.train calls attempt to access the model before reinitialization. This is only an issue when `fp16_full_eval` or `bf16_full_eval` flags are enabled.
* Update src/transformers/trainer.py
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com>
---------
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com>
2025-02-26 17:06:48 +01:00
Zach Mueller
41925e4213
Add retry hf hub decorator ( #35213 )
...
* Add retry torch decorator
* New approach
* Empty commit
* Empty commit
* Style
* Use logger.error
* Add a test
* Update src/transformers/testing_utils.py
Co-authored-by: Lucain <lucainp@gmail.com>
* Fix err
* Update tests/utils/test_modeling_utils.py
---------
Co-authored-by: Lucain <lucainp@gmail.com>
Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>
2025-02-25 20:53:11 +01:00
Chulhwa (Evan) Han
9ebfda3263
Fixed VitDet for non-squre Images ( #35969 )
...
* size tuple
* delete original input_size
* use zip
* process the other case
* Update src/transformers/models/vitdet/modeling_vitdet.py
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com>
* [VITDET] Test non-square image
* [Fix] Make Quality
* make fix style
* Update src/transformers/models/vitdet/modeling_vitdet.py
---------
Co-authored-by: Pavel Iakubovskii <qubvel@gmail.com>
2025-02-25 19:31:24 +00:00
Pavel Iakubovskii
fb83befb14
Fix pytorch integration tests for SAM ( #36397 )
...
Fix device in tests
2025-02-25 14:53:34 +00:00
jiqing-feng
7c8916ddb5
fix audio classification pipeline fp16 test on cuda ( #36359 )
...
* fix audio classification pipeline fp16 test on cuda
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
* fix format
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
* add comments
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
* Update tests/pipelines/test_pipelines_audio_classification.py
---------
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>
2025-02-25 15:01:25 +01:00
Fanli Lin
c3700b0eee
[tests] enable autoawq tests on XPU ( #36327 )
...
add autoawq
Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>
2025-02-25 13:38:09 +01:00
Dmitry Rogozhkin
b4b9da6d9b
tests: revert change of torch_require_multi_gpu to be device agnostic ( #35721 )
...
* tests: revert change of torch_require_multi_gpu to be device agnostic
The 11c27dd3311c27dd3311c27dd33#31098 )")
Signed-off-by: Dmitry Rogozhkin <dmitry.v.rogozhkin@intel.com>
* fix bug: modification of frozen set
---------
Signed-off-by: Dmitry Rogozhkin <dmitry.v.rogozhkin@intel.com>
Co-authored-by: Titus von Koeller <9048635+Titus-von-Koeller@users.noreply.github.com>
Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>
2025-02-25 13:36:10 +01:00
MAHIR DAIYAN
d80d52b007
addressing the issue #34611 to make FlaxDinov2 compatible with any batch size ( #35138 )
...
fixed the batch_size error, all tests are passing
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
2025-02-25 10:44:44 +00:00
jiqing-feng
9d6abf9778
enable torchao quantization on CPU ( #36146 )
...
* enable torchao quantization on CPU
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
* fix int4
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
* fix format
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
* enable CPU torchao tests
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
* fix cuda tests
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
* fix cpu tests
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
* update tests
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
* fix style
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
* fix cuda tests
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
* fix torchao available
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
* fix torchao available
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
* fix torchao config cannot convert to json
* fix docs
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
* rm to_dict to rebase
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
* limited torchao version for CPU
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
* fix format
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
* fix skip
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
* fix format
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
* Update src/transformers/testing_utils.py
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com>
* fix cpu test
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
* fix format
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
---------
Signed-off-by: jiqing-feng <jiqing.feng@intel.com>
Co-authored-by: Mohamed Mekkouri <93391238+MekkCyber@users.noreply.github.com>
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com>
2025-02-25 11:06:52 +01:00
Jerry Zhang
2af272c101
Add autoquant support for torchao quantizer ( #35503 )
...
* Add autoquant support for torchao quantizer
Summary:
att, also verified that autoquantized model can be saved and loaded:
save: https://gist.github.com/jerryzh168/01d367aaf44dbbbfd4068a4a10a00061
load: https://gist.github.com/jerryzh168/d5c6c401b2abdf18e0b6771341f1525c
Test Plan:
tested locally with above script
model uploaded to https://huggingface.co/jerryzh168/llama3-8b-autoquant
Reviewers:
Subscribers:
Tasks:
Tags:
* add test
* ruff fix
* ruff reformat
* add docs and min_sqnr support
* format
* format
* fix test
* update doc
* format
* remove disable_compile
* format
2025-02-24 15:54:16 +01:00
Rahul Tuli
884a8ea1f0
Improve model loading for compressed tensor models ( #36152 )
...
* Disable warnings for stacked compressors
* Introduce two new hooks in HfQuantizer lifecycle
to allow updates to missing and unexpected keys
* Update missing and unexpected keys
for stacked compressors
* Add tests
* Fix: run_compressed cases
* Fix: uncompressed cases
* Rename compressed_tensor folder to compressed_tensors
Move RunCompressedTest to the same file
Update tests to unittest
2025-02-24 13:47:21 +01:00
Fanli Lin
4dbf17c17f
[tests] enable bnb tests on xpu ( #36233 )
...
* fix failed test
* fix device
* fix more device cases
* add more cases
* fix empty cache
* Update test_4bit.py
---------
Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>
2025-02-24 11:30:15 +01:00
CalOmnie
547911e727
Uses Collection in transformers.image_transforms.normalize ( #36301 )
...
* Uses Collection instead of Sequence in transformers.image_transforms.normalize
* Uses collections.abc.Collection in lieu of deprecated typing one
2025-02-21 18:38:41 +01:00
Fanli Lin
7c5bd24ffa
[tests] make quanto tests device-agnostic ( #36328 )
...
* make device-agnostic
* name change
2025-02-21 14:20:40 +01:00
Joao Gante
678885bbbd
[CI] Check test if the GenerationTesterMixin inheritance is correct 🐛 🔫 ( #36180 )
2025-02-21 10:18:20 +00:00
Pavel Iakubovskii
a957b7911a
Add SigLIP 2 ( #36323 )
...
* Docs
* Inits
* Auto classes
* Add siglip base
* Add base tests
* Fix Siglip V1 for fix res version
* Add image processor
* Update conversion
* Experimenting with vectorized embeddings
* Fixup
* Add modular Siglip2Processor
* Add modular configuration
* Rename num patches
* Correct image and text features merging
* Working conversion script
* Refactoring conversion script
* Remove unused code in conversion script
* Shorten dict a bit
* Refactoring conversion
* Done conversion refactoring
* Fixup
* Modular siglip2
* Make model exportable and compilable without graph breaks
* Remove position_ids from image_processor
* REmove position ids from modeling file
* Update modular
* Type hint
* Fixup
* Set defaults to processor
* Add integration test
* Revert spatial shapes back to tensor
* Change order
* Fix most of the tests
* Fix docstring
* Remove interpolate_pos_encoding arg (not needed)
* Update docs
* Standardize processing
* Fix attention_mask in vision head
* Siglip v1: remove double transpose in FA2
* Update modular file
* Update FA2 test
* Update expected logits
* Fix interpolation for siglip2 image processor
* Skip init test
* Skip dispatch on flash test
* Fix modeling tests
* Fixup
* Add dummy objects
* Fix some docstrings
* Add siglip2 in index.md
* Fix consistency
* Add docs
* Remove size and data format
* Add image processor tests
* Fix
* Add fast image processor
* Fix style
* Fix
* Docs
* Set lowercase for tokenizer
* Adjust head size for Siglip v1
* Update siglip2 for consistency with siglip1
* Update siglip2 conversion
* Update pipeline
* Update checkpoints in tests
* Update checkpoint name
* Fix pooling for image classification model
* Fix FA2 test
* Update processor
* Fix check repo
* Update docs
* Fix typos
* Fix docstring for fast image processor
* Add siglip2 to FA2 docs
* Fix fast ip tests
* Fix constitency
* Fix tokenizer class for siglip v1
* Fix missing header
* Refactor scaling for clip, siglip, siglip2
* Remove unused imports
* Make fast IP default for siglip2
* Update docs
* Update checkpoints
* Update modular
* Update paper link
* Fixup
* Fix name in toctree
* Fix test
2025-02-21 09:04:19 +00:00
Nosimus
effaef334b
fix: prevent second save in the end of training if last step was saved already ( #36219 )
...
* fix: prevent second save in the end of training
* fix: prevent second save in the end of training
* test: added test for no duplicate save on epoch save strategy
* fix: removed TrainerControl
* chore: style formatting
---------
Co-authored-by: JaktensTid <jaktenstid1@gmail.com>
2025-02-20 17:38:52 +01:00
Orr Zohar
4397dfcb71
SmolVLM2 ( #36126 )
...
* smolvlm init
* updates
* fixing bugs
* minimal run, no checks
* minimal run, no checks
* passing first check + adding url support
* updating video dataloading logic
* fixing image logic
* trying modular, but fails
* modular is working, changing processor to match PR comments and general transformers logic
* fixing kwargs
* offloading video loading logic to image_util
* fixing circleci code formatting errors
* fixing circleci code formatting errors
* fixing circleci code formatting errors
* fixing circleci code formatting errors
* fixing circleci code formatting errors
* fixing circleci code formatting errors
* fixing circleci code formatting errors
* fixing circleci code formatting errors
* fixing circleci code formatting errors
* fixing circleci code formatting errors
* fixing circleci code formatting errors
* fixing circleci code formatting errors
* fixing circleci code formatting errors
* fixing circleci code formatting errors
* update
* add idefics3-based tests
* add keyword to all
* add PreTrainedModel
* updateing video loading logic
* working inference
* updates for PR comments
* updates for PR comments
* moving SmolVLMPretrainedModel higher to fix import error
* CI test pass
* CI test pass
* removing lambda
* CI test pass
* CI test pass
* CI test pass
* CI test pass
* CI test pass
* CI test pass
* processor tests
* add example in docs
* typo
* fix copies
* skip compile tests - sdpa for VisionTransformer
* fix init
* raise import error for num2words
* update doc for FA2
* more doc fix
* CI
* updates for PR comments
* Update docs/source/en/model_doc/smolvlm.md
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
* Update docs/source/en/model_doc/smolvlm.md
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
* Update docs/source/en/model_doc/smolvlm.md
Co-authored-by: Joshua Lochner <admin@xenova.com>
* Update docs/source/en/model_doc/smolvlm.md
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
* Update docs/source/en/model_doc/smolvlm.md
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
* fixing processor -- tokenizer not defined properly, (gpt2 tokenizer), and does not have the attributes of fake image token, etc
* adding smolvlm to VQA models
* removing vqa auto class
* Update src/transformers/models/smolvlm/processing_smolvlm.py
Co-authored-by: Joshua Lochner <admin@xenova.com>
* removing smolvlmvisiontransformer from index.md
* my bad, video processing had typos
* fixing docs
* renaming params in SmolVLMModel.inputs_merger
* removing un-needed dtype/device in model forward
* ruff for CI
* update docs
* Update docs/source/en/model_doc/smolvlm.md
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
* return cache position
* return cache position
* return cache also in modular
* needed to run modular again
* fix training tests
* push vectorized inputs merger
* format
* format
* reduce number of mappings
* addressing PR comments
* happy CI, happy me :)
* skip non-nested images
* adjust integration test for smaller GPUs
* format
* fix kwargs in chat template apply
* skip this for now
---------
Co-authored-by: raushan <raushan@huggingface.co>
Co-authored-by: Pablo <pablo.montalvo.leroux@gmail.com>
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
Co-authored-by: Joshua Lochner <admin@xenova.com>
2025-02-20 15:00:26 +01:00
Yih-Dar
e8531a0e33
Fix broken CI on release branch due to missing conversion files ( #36275 )
...
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2025-02-20 13:22:10 +01:00
Ilyas Moutawwakil
5e2183f344
Make cache traceable ( #35873 )
...
simply make cache traceable
2025-02-20 09:59:25 +01:00
Joao Gante
e3d99ec2f5
[tests] make test_from_pretrained_low_cpu_mem_usage_equal less flaky ( #36255 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2025-02-19 15:14:02 +00:00
Joao Gante
99adc74462
[tests] remove flax-pt equivalence and cross tests ( #36283 )
2025-02-19 15:13:27 +00:00
Joao Gante
fa8cdccd91
[tests] deflake dither test ( #36284 )
2025-02-19 15:13:10 +00:00
Cyril Vallez
60226c6ff3
TP initialization module-by-module ( #35996 )
...
* module-by-module loading!
* Update modeling_utils.py
* dtyle and comments
* Update modeling_utils.py
* Update modeling_utils.py
* Update test
* Update modeling_utils.py
* Update modeling_utils.py
* Update test_tp.py
* Update test_tp.py
* Update modeling_utils.py
* re-trigger CIs
* re-trigger CIs
2025-02-19 14:04:57 +01:00
Joao Gante
0863eef248
[tests] remove pt_tf equivalence tests ( #36253 )
2025-02-19 11:55:11 +00:00
Karel Vesely
1a81d774b1
Add dithering to the Speech2TextFeatureExtractor API. ( #34638 )
...
* Add dithering to the `Speech2TextFeatureExtractor` API.
- in kaldi : 4a8b7f6732/src/feat/feature-window.cc (L145)
2025-02-19 11:50:02 +01:00
Yoni Gozlan
9f51dc2535
Add support for post-processing kwargs in image-text-to-text pipeline ( #35374 )
...
* fix error and improve pipeline
* add processing_kwargs to apply_chat_template
* change default post_process kwarg to args
* Fix slow tests
* fix copies
2025-02-18 17:43:36 -05:00
Yoni Gozlan
9b479a245b
Uniformize LlavaNextVideoProcessor kwargs ( #35613 )
...
* Uniformize processor kwargs and add tests
* add videos_kwargs tests
* fix copies
* fix llava_next_video chat template tests
* remove unnecessary default kwargs
2025-02-18 14:13:51 -05:00
ivarflakstad
07182b2e10
GitModelIntegrationTest - flatten the expected slice tensor ( #36260 )
...
Flatten the expected slice tensor
2025-02-18 16:04:19 +01:00
Damiano Amatruda
4d2de5f63c
Fix XGLM loss computation (PyTorch and TensorFlow) ( #35878 )
...
* Fix XGLM loss computation (PyTorch and TensorFlow)
* Update expected output string in XGLM sample test
This updates the expected output string of test_xglm_sample for torch
2.0 to the correct one and removes the one for torch 1.13.1 + cu116
(transformers moved to torch 2.0 with PR #35358 ).
* Update expected output IDs in XGLM generation test
2025-02-18 15:37:48 +01:00
Raushan Turganbay
e6cc410d5b
Remove flakiness in VLMs ( #36242 )
...
* fix
* nit
* no logits processor needed
* two more tests on assisted decoding
2025-02-18 11:41:07 +01:00
andrewor14
fdcfdbfd22
Fix TorchAoConfig not JSON serializable ( #36206 )
...
**Summary:** TorchAoConfig optionally contains a
`torchao.dtypes.Layout` object which is a dataclass and not
JSON serializable, and so the following fails:
```
import json
from torchao.dtypes import TensorCoreTiledLayout
from transformers import TorchAoConfig
config = TorchAoConfig("int4_weight_only", layout=TensorCoreTiledLayout())
config.to_json_string()
json.dumps(config.to_dict())
```
This also causes `quantized_model.save_pretrained(...)` to
fail because the first step of this call is to JSON serialize
the config. Fixes https://github.com/pytorch/ao/issues/1704 .
**Test Plan:**
python tests/quantization/torchao_integration/test_torchao.py -k test_json_serializable
Co-authored-by: Mohamed Mekkouri <93391238+MekkCyber@users.noreply.github.com>
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com>
2025-02-18 11:05:42 +01:00
Yih-Dar
626666c444
Au revoir flaky test_fast_is_faster_than_slow ( #36240 )
...
* fix
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2025-02-17 18:30:07 +01:00
Joao Gante
429f1a682d
[tests] remove test_export_to_onnx ( #36241 )
2025-02-17 16:52:44 +00:00
Joao Gante
55493f1390
[tests] remove tf/flax tests in /generation ( #36235 )
2025-02-17 14:59:22 +00:00
ivarflakstad
7ec35bc3bd
Add missing atol to torch.testing.assert_close where rtol is specified ( #36234 )
2025-02-17 14:57:50 +01:00
