riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-19 20:48:22 +06:00
Code Issues Actions 18 Packages Projects Releases Wiki Activity
19,383 Commits 876 Branches 227 Tags 2 GiB
remove-script-datasets-in-tests-test-datasets-main
Commit Graph

19383 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Sungmin Oh
cd9a3c49b8
🌐 [i18n-KO] Translated model_doc/dbrx.md to Korean (#33951) 
* docs: ko: model_doc/dbrx.md

* feat: nmt draft

* fix: resolve suggestions

Co-authored-by: SeongWooChoi <46990061+nuatmochoi@users.noreply.github.com>

* fix: resolve suggestions

* fix: resolve suggestions

---------

Co-authored-by: SeongWooChoi <46990061+nuatmochoi@users.noreply.github.com>
 2024-10-08 17:14:42 -07:00
Sungmin Oh
d6d07f9c77
🌐 [i18n-KO] Translated model_doc/cohere.md to Korean (#33885) 
* docs: ko: model_doc/cohere.md

* feat: nmt draft

* fix: resolve suggestions

Co-authored-by: HyeokJun SHIN <96534680+jun048098@users.noreply.github.com>
Co-authored-by: SeongWooChoi <46990061+nuatmochoi@users.noreply.github.com>

* fix: resolve suggestions

---------

Co-authored-by: HyeokJun SHIN <96534680+jun048098@users.noreply.github.com>
Co-authored-by: SeongWooChoi <46990061+nuatmochoi@users.noreply.github.com>
 2024-10-08 17:14:25 -07:00
Sungmin Oh
48e80284fa
🌐 [i18n-KO] Translated model_doc/mistral.md to Korean (#33648) 
* docs: ko: model_doc/mistral.md

* feat: nmt draft

* fix: resolve suggestions

Co-authored-by: Ahnjj_DEV <ahnjj.dev@gmail.com>
Co-authored-by: Chaewon Song <chaewon1019@ewhain.net>
Co-authored-by: HyeokJun SHIN <96534680+jun048098@users.noreply.github.com>

* fix: resolve suggestions

* fix: resolve suggestions

Co-authored-by: HyeokJun SHIN <96534680+jun048098@users.noreply.github.com>

---------

Co-authored-by: Ahnjj_DEV <ahnjj.dev@gmail.com>
Co-authored-by: Chaewon Song <chaewon1019@ewhain.net>
Co-authored-by: HyeokJun SHIN <96534680+jun048098@users.noreply.github.com>
 2024-10-08 17:14:12 -07:00
Sungmin Oh
adb14b93f4
🌐 [i18n-KO] Translated model_doc/llama3.md to Korean (#33635) 
* docs: ko: model_doc/llama3.md

* fix: resolve suggestions

* fix: resolve suggestions

Co-authored-by: Chaewon Song <chaewon1019@ewhain.net>

* fix: resolve suggestions

Co-authored-by: HyeokJun SHIN <96534680+jun048098@users.noreply.github.com>

* fix: resolve suggestions

* fix: resolve suggestions

Co-authored-by: Chaewon Song <chaewon1019@ewhain.net>

* fix: resolve suggestions

Co-authored-by: Ahnjj_DEV <ahnjj.dev@gmail.com>

* fix: resolve suggestions

Co-authored-by: Ahnjj_DEV <ahnjj.dev@gmail.com>

* fix: resolve suggestions

---------

Co-authored-by: Chaewon Song <chaewon1019@ewhain.net>
Co-authored-by: HyeokJun SHIN <96534680+jun048098@users.noreply.github.com>
Co-authored-by: Ahnjj_DEV <ahnjj.dev@gmail.com>
 2024-10-08 17:13:57 -07:00
Sungmin Oh
291e707868
🌐 [i18n-KO] Translated model_doc/paligemma.md to Korean (#33612) 
* docs: ko: model_doc/paligemma.md

* feat: nmt draft

* fix: resolve suggestions

Co-authored-by: Ahnjj_DEV <ahnjj.dev@gmail.com>

* fix: resolve suggestions

* fix: resolve suggestions

Co-authored-by: Ahnjj_DEV <ahnjj.dev@gmail.com>

* fix: resolve suggestions

* fix: resolve suggestions

---------

Co-authored-by: Ahnjj_DEV <ahnjj.dev@gmail.com>
 2024-10-08 17:13:25 -07:00
Sungmin Oh
dd43dafa39
🌐 [i18n-KO] Translated model_doc/clip.md to Korean (#33610) 
* docs: ko: model_doc/clip.md

* feat: nmt draft

* fix: manual edits

* fix: resolve suggestions

Co-authored-by: Ahnjj_DEV <ahnjj.dev@gmail.com>

* fix: resolve suggestions

Co-authored-by: HyeokJun SHIN <96534680+jun048098@users.noreply.github.com>

* fix: resolve suggestions

Co-authored-by: Ahnjj_DEV <ahnjj.dev@gmail.com>

* fix: resolve suggestions

Co-authored-by: Ahnjj_DEV <ahnjj.dev@gmail.com>

* fix: resolve suggestions

Co-authored-by: Ahnjj_DEV <ahnjj.dev@gmail.com>

* fix: resolve suggestions

Co-authored-by: Ahnjj_DEV <ahnjj.dev@gmail.com>

* fix: resolve suggestions

Co-authored-by: Ahnjj_DEV <ahnjj.dev@gmail.com>

* fix: resolve suggestions

Co-authored-by: HyeokJun SHIN <96534680+jun048098@users.noreply.github.com>

* fix: resolve suggestions

* fix: resolve suggestions

* fix: resolve suggestions

* fix: resolve suggestions

Co-authored-by: Ahnjj_DEV <ahnjj.dev@gmail.com>

* fix: resolve suggestions

* fix: resolve suggestions

Co-authored-by: Ahnjj_DEV <ahnjj.dev@gmail.com>

* fix: resolve suggestions

---------

Co-authored-by: Ahnjj_DEV <ahnjj.dev@gmail.com>
Co-authored-by: HyeokJun SHIN <96534680+jun048098@users.noreply.github.com>
 2024-10-08 17:13:07 -07:00
Sungmin Oh
acde6c7d9d
🌐 [i18n-KO] Translated model_doc/patchtsmixer.md to Korean (#33587) 
* docs: ko: model_doc/patchtsmixer.md

* feat: nmt draft

* fix: manual edits

* fix: resolve suggestions

Co-authored-by: HyeokJun SHIN <96534680+jun048098@users.noreply.github.com>

* fix: resolve suggestions

---------

Co-authored-by: HyeokJun SHIN <96534680+jun048098@users.noreply.github.com>
 2024-10-08 17:11:48 -07:00
Sungmin Oh
bb825dde73
🌐 [i18n-KO] Translated model_doc/autoformer.md to Korean (#33574) 
* docs: ko: model_doc/autoformer.md

* feat: nmt draft

* fix: manual edits

* fix: resolve suggestions
 2024-10-08 17:11:19 -07:00
Sungmin Oh
1d458437dd
🌐 [i18n-KO] Translated model_doc/mamba.md to Korean (#33626) 
* docs: ko: model_doc/mamba.md

* fix: resolve suggestions

Co-authored-by: Ahnjj_DEV <ahnjj.dev@gmail.com>

* fix: resolve suggestions

* fix: resolve suggestions

---------

Co-authored-by: Ahnjj_DEV <ahnjj.dev@gmail.com>
 2024-10-08 17:11:11 -07:00
Sungmin Oh
47da2c528b
🌐 [i18n-KO] Translated main_classes/configuration.md to Korean (#33952) 
* docs: ko: main_classes/configuration.md

* feat: nmt draft
 2024-10-08 17:11:02 -07:00
Sungmin Oh
2e8de976bd
🌐 [i18n-KO] Translated main_classes/quantization.md to Korean (#33959) 
* docs: ko: main_classes/quantization.md

* feat: nmt draft

* fix: resolve suggestions

Co-authored-by: Ahnjj_DEV <ahnjj.dev@gmail.com>

* fix: resolve suggestions

Co-authored-by: Ahnjj_DEV <ahnjj.dev@gmail.com>

* fix: resolve suggestions

---------

Co-authored-by: Ahnjj_DEV <ahnjj.dev@gmail.com>
 2024-10-08 17:10:41 -07:00
Chaewon Song
2fe77783c3
🌐 [i18n-KO] Translated rag.md to Korean (#33989) 
* fix: toctree edits

* feat: nmt-draft

* fix: edit Inline TOC
 2024-10-08 17:10:26 -07:00
Ahnjj_DEV
1ed98773e5
🌐 [i18n-KO] Translated gpt_neox_japanese.md to Korean (#33894) 
* docs: ko: gpt_neox_japanese.md

* Update _toctree.yml

* fix: manual edits

* Update docs/source/ko/model_doc/gpt_neox_japanese.md

Co-authored-by: Sungmin Oh <fabxoe.kor@gmail.com>

* Update docs/source/ko/model_doc/gpt_neox_japanese.md

Co-authored-by: Sungmin Oh <fabxoe.kor@gmail.com>

* Update docs/source/ko/model_doc/gpt_neox_japanese.md

Co-authored-by: Sungmin Oh <fabxoe.kor@gmail.com>

---------

Co-authored-by: Sungmin Oh <fabxoe.kor@gmail.com>
 2024-10-08 17:08:06 -07:00
Ahnjj_DEV
79af52ad9a
🌐 [i18n-KO] Translated bertweet.md to Korean (#33891) 
* docs: ko: bertweet.md

* Update _toctree.yml

* fix: manual edits

* Update docs/source/ko/model_doc/bertweet.md

Co-authored-by: HyeokJun SHIN <96534680+jun048098@users.noreply.github.com>

---------

Co-authored-by: HyeokJun SHIN <96534680+jun048098@users.noreply.github.com>
 2024-10-08 17:07:13 -07:00
Yijun Lee
d49999ce11
🌐 [i18n-KO] Translated feature_extractor.md to Korean (#33775) 
* docs: ko: feature_extractor.md

* feat: nmt draft

* fix: manual edits
 2024-10-08 17:06:56 -07:00
Ben Lewis
573942d96a
Fix trainer_seq2seq.py's __init__ type annotations (#34021) 
* Fix `trainer_seq2seq.py`'s `__init__` type annotations

* Update src/transformers/trainer_seq2seq.py

Co-authored-by: Lysandre Debut <hi@lysand.re>

* Fix issue pointed out by `muellerzr`

---------

Co-authored-by: Lysandre Debut <hi@lysand.re>
 2024-10-08 16:43:30 -04:00
Marc Sun
04b4e441dc
Remove decoder_config=None (#34014) 
* remove unnecessary line

* changed to the right one
 2024-10-08 15:57:12 +02:00
Marc Sun
1909def2de
fix awq tests due to ipex backend (#34011) 
fix awq tests
 2024-10-08 15:56:05 +02:00
Marc Sun
4f2bf135af
Fix typing issue (#34012) 2024-10-08 15:15:40 +02:00
Zach Mueller
f4b741d674
Fixup DeepSpeed things (#34007) 2024-10-08 09:04:24 -04:00
Cyril Vallez
17806d11ba
Improve modular converter (#33991) 
* improve modular

* style

* Update modular_model_converter.py

* pretty print warning

* style

* Support to remove unused classes as part of added dependencies as well

* nits

* correct bug

* add example

* style

* Add documentation
 2024-10-08 14:53:58 +02:00
Matt
fb360a6c7a
BatchFeature.to() supports non-tensor keys (#33918) 
* Fix issue in oneformer preprocessing

* [run slow] oneformer

* [run_slow] oneformer

* Make the same fixes in DQA and object detection pipelines

* Fix BatchFeature.to() instead

* Revert pipeline-specific changes

* Add the same check in Pixtral's methods

* Add the same check in BatchEncoding

* make sure torch is imported
 2024-10-08 13:43:32 +01:00
Matt
3b44d2f042
Image pipelines spec compliance (#33899) 
* Update many similar visual pipelines

* Add input tests

* Add ImageToText as well

* Add output tests

* Add output tests

* Add output tests

* OutputElement -> Output

* Correctly test elements

* make fixup

* fix typo in the task list

* Fix VQA testing

* Add copyright to image_classification.py

* Revert changes to VQA pipeline because outputs have differences - will move to another PR

* make fixup

* Remove deprecation warnings
 2024-10-08 13:34:28 +01:00
Yoni Gozlan
e2001c3413
Add auto model for image-text-to-text (#32472) 
* Add Auto model for image-text-to-text

* Remove donut from processing auto, add chameleon ti image text to text models

* add qwen2_vl and llava_onevision

* add pixtral to auto model for image-text-to-text

* add mllama and idefics3

* remove models in IGNORE_NON_AUTO_CONFIGURED

* add AutoModelForImageTextToText to tests and doc
 2024-10-08 14:26:43 +02:00
Raushan Turganbay
0dbc7090ba
Processors: don't default padding side (#33942) 
* don't default padding side

* fix
 2024-10-08 10:58:49 +02:00
Arthur
a3add29097
Add support for __all__ and potentilly deleting functions (#33859) 
* Add support for __all__ and potentailly deleting functions

* updates

* update

* nits

* remove dummies

* fix warning

* fixup

* style

* update

* fixup

* skip copied from when # skip

* remove log

* bring dummies back

* fixup

* remove copied from

* fixup

* remove warnings from `make fix-copies`

* fix doc issues

* nits

* Better error message !

* add support for more flexible naming!

* style

* breaking style?

* fix super() renaming issues

* del not needed when you don't call super().__init__()

* style

* no more fmt on :)

* properly remove `self`

* fixup

* fix

* doc nits

* add some doc 🫡
 2024-10-08 10:19:17 +02:00
Raushan Turganbay
bead0fa8dc
Cache: slight change in naming (#32421) 
* squash

* codestyle

* Update src/transformers/cache_utils.py

Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>

* propagate changes to all cache classes

* + whisper

* fix tests

* more fixes

* add deprecation warning

* fix copies

* address comments

* fix mistral also

* these didn't have "copied from"

---------

Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
 2024-10-08 09:43:40 +02:00
Yijun Lee
d6ba1ac041
🌐 [i18n-KO] Translated gemma.md to Korean (#33936) 
* docs: ko: gemma.md

* feat: nmt draft

* fix: manual edits
 2024-10-07 15:59:14 -07:00
Jiwook Han
46f146a2b5
🌐 [i18n-KO] Translated vit.md to Korean (#33884) 
* docs: ko: model_doc/vit.md

* feat: nmt draft

* fix: manual edits

* fix: manual edits

* Update docs/source/ko/model_doc/vit.md

Co-authored-by: Yijun Lee <119404328+yijun-lee@users.noreply.github.com>

* Update docs/source/ko/model_doc/vit.md

Co-authored-by: Chulhwa (Evan) Han <cjfghk5697@ajou.ac.kr>

---------

Co-authored-by: Yijun Lee <119404328+yijun-lee@users.noreply.github.com>
Co-authored-by: Chulhwa (Evan) Han <cjfghk5697@ajou.ac.kr>
 2024-10-07 15:35:11 -07:00
Jiwook Han
1ecca92f03
🌐 [i18n-KO] Translated swin2sr.md to Korean (#33795) 
* ko: doc: model_doc/swin2sr.md

* feat: nmt draft

* Update docs/source/ko/model_doc/swin2sr.md

Co-authored-by: Yijun Lee <119404328+yijun-lee@users.noreply.github.com>

---------

Co-authored-by: Yijun Lee <119404328+yijun-lee@users.noreply.github.com>
 2024-10-07 15:34:56 -07:00
boyunJang
8258219c4c
🌐 [i18n-KO] Translated auto.md to Korean (#33590) 
* docs: ko: model_doc/auto.md

* feat: nmt draft

* fix: manual edits

* fix: resolve suggestions

Co-authored-by: wony617 <49024958+Jwaminju@users.noreply.github.com>
Co-authored-by: YONGSANG <71686691+4N3MONE@users.noreply.github.com>

* fix: resolve suggestions

---------

Co-authored-by: wony617 <49024958+Jwaminju@users.noreply.github.com>
Co-authored-by: YONGSANG <71686691+4N3MONE@users.noreply.github.com>
 2024-10-07 15:34:45 -07:00
Chaewon Song
253a9a9d6f
🌐 [i18n-KO] Translated logging.md to Korean (#33543) 
* docs: ko: main_classes/logging.md

* feat: nmt-draft

* fix: update toctree.yml

* Update docs/source/ko/main_classes/logging.md

Co-authored-by: Sungmin Oh <fabxoe.kor@gmail.com>

* Update docs/source/ko/main_classes/logging.md

Co-authored-by: HyeokJun SHIN <96534680+jun048098@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: HyeokJun SHIN <96534680+jun048098@users.noreply.github.com>
Co-authored-by: Sungmin Oh <fabxoe.kor@gmail.com>

* Apply suggestions from code review

Co-authored-by: Ahnjj_DEV <ahnjj.dev@gmail.com>

---------

Co-authored-by: Sungmin Oh <fabxoe.kor@gmail.com>
Co-authored-by: HyeokJun SHIN <96534680+jun048098@users.noreply.github.com>
Co-authored-by: Ahnjj_DEV <ahnjj.dev@gmail.com>
 2024-10-07 15:34:34 -07:00
Yijun Lee
178d707b7e
🌐 [i18n-KO] Translated chameleon.md to Korean (#33799) 
* docs: ko: chameleon.md

* feat: nmt draft

* fix: manual edits

* fix: resolve suggestions

Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com>
Co-authored-by: Chulhwa (Evan) Han <cjfghk5697@ajou.ac.kr>

---------

Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com>
Co-authored-by: Chulhwa (Evan) Han <cjfghk5697@ajou.ac.kr>
 2024-10-07 15:06:13 -07:00
Yijun Lee
13432f8409
🌐 [i18n-KO] Translated trainer.md to Korean (#33797) 
* docs: ko: trainer.md

* feat: nmt draft

* fix: manual edits

* fix: resolve suggestions

Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com>
Co-authored-by: Chulhwa (Evan) Han <cjfghk5697@ajou.ac.kr>

---------

Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com>
Co-authored-by: Chulhwa (Evan) Han <cjfghk5697@ajou.ac.kr>
 2024-10-07 15:05:57 -07:00
Yijun Lee
e9fbe62965
🌐 [i18n-KO] Translated pipelines_utils.md to Korean (#33809) 
* docs: ko: pipelines_utils.md

* feat: nmt draft

* fix: manual edits
 2024-10-07 15:05:17 -07:00
Yijun Lee
9c61ba2f25
🌐 [i18n-KO] Translated time_series_utils.md to Korean (#33806) 
* docs: ko: time_series_utils.md

* feat: nmt draft

* fix: manual edits
 2024-10-07 15:05:00 -07:00
Yijun Lee
9c8bd3fc1b
🌐 [i18n-KO] Translated esm.md to Korean (#33796) 
* docs: ko: esm.md

* feat: nmt draft

* fix: manual edits
 2024-10-07 13:39:22 -07:00
Yijun Lee
6996f2186a
🌐 [i18n-KO] Translated audio_utils.md to Korean (#33802) 
* docs: ko: audio_utils.md

* feat: nmt draft

* fix: manual edits
 2024-10-07 13:39:10 -07:00
Jiwook Han
410c73af1d
🌐 [i18n-KO] Translated swinv2.md to Korean (#33566) 
* docs: ko: model_doc/swinv2.md

* feat: nmt draft

* fix: manual edits

* fix: manual edits
 2024-10-07 12:50:43 -07:00
Yijun Lee
6c18cefed0
🌐 [i18n-KO] Translated gguf.md to Korean (#33764) 
* docs: ko: gguf.md

* feat nmt draft

* fix: manual edits

* fix: resolve suggestions

Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com>
Co-authored-by: Chulhwa (Evan) Han <cjfghk5697@ajou.ac.kr>

---------

Co-authored-by: Jiwook Han <33192762+mreraser@users.noreply.github.com>
Co-authored-by: Chulhwa (Evan) Han <cjfghk5697@ajou.ac.kr>
 2024-10-07 12:49:08 -07:00
Michael Goin
c91fe85b78
Fix undefined default_config in configuration_utils.py (#33934) 2024-10-07 18:32:20 +02:00
Arthur
736c7cde51
[pytes collection] Fix flax test collection (#34004) 
bit weird but to filter I had to use this
 2024-10-07 18:11:13 +02:00
roy
55be7c4c48
Enable customized optimizer for DeepSpeed (#32049) 
* transformers: enable custom optimizer for DeepSpeed

* transformers: modify error message

---------

Co-authored-by: datakim1201 <roy.kim@maum.ai>
 2024-10-07 15:36:54 +02:00
Arthur
7bae833728
properly fix and RUN_SLOW (#33965) 
* properly fix and RUN_SLOW

* lots of models were affected

* fix-copies

* more fixes
 2024-10-07 14:45:57 +02:00
Kaito
e782e95e34
Fix Tensor + Embedding error in some cases when using SiglipVisionModel (#33994) 
Fix Tensor + Embedding error in some cases

Co-authored-by: kaitolucifer <kaito.o@ghelia.com>
 2024-10-07 11:17:34 +02:00
Arthur
9b4b0c07db
[Red CIs] Fix hub failures (#34001) 
maybe setup should work?
 2024-10-07 10:56:24 +02:00
Magnus
ad1a250719
[Docs] Add Developer Guide: How to Hack Any Transformers Model (#33979) 
* docs: add example for separating q, k, v projections in SAM

* docs: How to Hack Any Transformers Model

* docs: remove changes from sam model docs

* Apply suggestions from code review

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

---------

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
 2024-10-07 10:08:20 +02:00
NielsRogge
f5aeb7c1a5
[Docs] Improve VLM docs (#33393) 
* Improve docs

* Update docs/source/en/model_doc/llava.md

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* Update docs/source/en/model_doc/llava.md

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* Address comment

* Address comment

* Improve pixtral docs

---------

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2024-10-07 09:54:07 +02:00
Cyril Vallez
1f33023cfa
Flash-attn performance: remove cuda sync during inference (#33570) 
Switch conditions to use short-circuit during inference
 2024-10-07 09:52:19 +02:00
Avishai Elmakies
4953ddf036
Add position ids in forward pass to opt model (#33121) 
* start working on adding position ids

* add docs

* Refactor modeling_biogpt.py and modeling_opt.py for code consistency

* fix 2 PR comments

* move position_ids to end of args

* remove trailing white space

* add comment with TODO

* bug fix gradient checkpointing

* fixup

* missed on position_ids

* remove _attention_to_position_ids and refactor embedding class

* remove redundent code

---------

Co-authored-by: Avishai Elmakies <avishai.elma@cs.huji.ac.il>
 2024-10-07 09:20:49 +02:00