* Gemma: only display act. warning when necessary
This is a nit PR, but I was confused. I got the warning even after I
had changed `hidden_act` to `gelu_pytorch_tanh`, telling me that I
was using the "legacy" `gelu_pytorch_tanh`.
Another option is to keep the warning but change the message to say
something like "`hidden_act` is ignored, please use `hidden_activation`
instead. Setting Gemma's activation function to `gelu_pytorch_tanh`".
* Change message, and set `config.hidden_activation`
* Temporarily silence warnings in apply_chat_template until we can properly deprecate default chat templates
* make fixup
* Move the default chat template warning into apply_chat_template itself
* make fixup
* move scaling to nn.Module
* let the test be here for now (need to fix)
* failing tests
* last failing models
* Revert commit 4c14817f38
* clean-up
* oops forgot
* codestyle
* raise NotImplemented when possible
* Update tests/test_modeling_common.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* skip tests in respective modeling files
---------
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* fix seq2seq data collator to respect the given padding strategy
further added tests for the seq2seq data collator in the style of the `data_collator_for_token_classification` (pt, tf, np)
* formatting and change bool equals "==" to "is"
* add missed return types in tests
* update numpy test as it can handle unequal shapes, not like pt or tf
* feat: support for dinov2
* feat: support for depth_anything
* feat: support for efficientformer
* feat: support for bert (is this right?)
* update: embedding split
* remove: empty string
* feat: support for align
* fix: copies
* fix: QAQBertEmbeddings
* fix: more consistency issues
* revert: support for effientformer
* feat: support for altclip
* feat: support for blip_text
* support for ChineseCLIP
* feat: support for depth anything
* feat: support for dpt
* feat: support for dpt
* feat: support for git
* feat: support for groupvit
* update: format
* fix: support for clip
* fix: consistency
* feat: support for pvt
* feat: support for vit_msn
* fix: consistency
* fix: other copies
* remove: device transfer
* revert: in-place add
* update: support for align
* update: support for bert
* update: support for Chinese CLIP
* revert: changes to efficientformer
* update: support for dpt
* update: support for efficientformer
* revert: changes to git
* revert: changes to groupvit
* revert: changes to roc_bert
* update: support for vit_msn
* revert: changes to dpt
* remove: extra space
* style: extra space
* Reenable SDPA's FA2 during training with torch.compile
* fix Olmo's SDPA FA2 dispatching too
* update formatting
* improved SDPA comment
* formatting and explanatory comment
* is_causal if statement to one-liner
* Enable instantiating model with pretrained backbone weights
* Clarify pretrained import
* Use load_backbone instead
* Add backbone_kwargs to config
* Fix up
* Add tests
* Tidy up
* Enable instantiating model with pretrained backbone weights
* Update tests so backbone checkpoint isn't passed in
* Clarify pretrained import
* Update configs - docs and validation check
* Update src/transformers/utils/backbone_utils.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* Clarify exception message
* Update config init in tests
* Add test for when use_timm_backbone=True
* Use load_backbone instead
* Add use_timm_backbone to the model configs
* Add backbone_kwargs to config
* Pass kwargs to constructors
* Draft
* Fix tests
* Add back timm - weight naming
* More tidying up
* Whoops
* Tidy up
* Handle when kwargs are none
* Update tests
* Revert test changes
* Deformable detr test - don't use default
* Don't mutate; correct model attributes
* Add some clarifying comments
* nit - grammar is hard
---------
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* Adding SDPA support for BERT
* Using the proper input name for testing model input in inference()
* Adding documentation for SDPA in BERT model page
* Use the stable link for the documentation
* Adding a gate to only call .contiguous() for torch < 2.2.0
* Additions and fixes to the documentation
* Minor updates to documentation
* Adding extra requirements needed for the contiguous() bug
* Adding "Adapted from" in plcae of the "Copied from"
* Add benchmark speedup tables to the documentation
* Minor fixes to the documentation
* Use ClapText as a replacemenet for Bert in the Copied-From
* Some more fixes for the fix-copies references
* Overriding the test_eager_matches_sdpa_generate in bert tests to not load with low_cpu_mem_usage
[test all]
* Undo changes to separate test
* Refactored SDPA self attention code for KV projections
* Change use_sdpa to attn_implementation
* Fix test_sdpa_can_dispatch_on_flash by preparing input (required for MultipleChoice models)
* Draft tutorial for talking to chat models
* Reformat lists and text snippets
* Cleanups and clarifications
* Finish up remaining TODOs
* Correct section link
* Small fix
* Add proper quantization examples
* Add proper quantization examples
* Add proper quantization examples
* Update docs/source/en/conversations.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/conversations.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/conversations.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/conversations.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/conversations.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/conversations.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/conversations.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/conversations.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/conversations.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/conversations.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/_toctree.yml
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/conversations.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Fix Text Generation Pipeline link and add a ref to the LLM inference guide
* intelligent -> capable
* Small intro cleanup
* Small text cleanup
* Small text cleanup
* Clarification about system message
* Clarification about system message
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Introduce saveable callbacks
* Add note
* Test for non-present and flag
* Support early stopping and refusing to train further
* Update docstring
* More saving
* Import oopsie
* Apply suggestions from code review
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Make it go through TrainerArguments
* Document
* Fix test
* Apply suggestions from code review
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Rework to allow for duplicates
* CLean
* Fix failing tests
---------
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
