riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-31 02:02:21 +06:00
Code Issues Actions 23 Packages Projects Releases Wiki Activity
13,677 Commits 887 Branches 232 Tags 2.1 GiB
5e5fa0d88c
Commit Graph

13677 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
amyeroberts
5e5fa0d88c
Mark flaky tests (#25463) 
Make CI less brittle
 2023-08-11 15:26:45 +01:00
amyeroberts
11757e2bbd
Add input_data_format argument, image transforms (#25462) 
* Enable specifying input data format - overriding inferring

* Add tests
 2023-08-11 15:09:31 +01:00
Gema Parreño
0acf56224b
Update run_translation.py broken link example Pytoch (#25461) 
* Update run_translation.py

Fixed link

* Update run_translation.py
 2023-08-11 15:41:24 +02:00
Yih-Dar
1d75768695
Reuse the cache created for latest main on PRs/branches if setup.py is not modified (#25445) 
* fix

* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-08-11 14:40:51 +02:00
Joao Gante
4692d26194
Switch Transformers: remove overwritten beam sample test (#25458) 2023-08-11 13:16:01 +01:00
amyeroberts
41d56ea6dd
Refactor image processor testers (#25450) 
* Refactor image processor test mixin

- Move test_call_numpy, test_call_pytorch, test_call_pil to mixin
- Rename mixin to reflect handling of logic more than saving
- Add prepare_image_inputs, expected_image_outputs for tests

* Fix for oneformer
 2023-08-11 11:30:18 +01:00
Yih-Dar
454957c9bb
Fix for #25437 (#25454) 
* fix

* fix

* fix

* fix

* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-08-11 11:39:57 +02:00
Marc Sun
55db70c63d
GPTQ integration (#25062) 
* GTPQ integration

* Add tests for gptq

* support for more quantization model

* fix style

* typo

* fix method

* Update src/transformers/modeling_utils.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* add dataclass and fix quantization_method

* fix doc

* Update tests/quantization/gptq/test_gptq.py

Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>

* modify dataclass

* add gtpqconfig import

* fix typo

* fix tests

* remove dataset as req arg

* remove tokenizer import

* add offload cpu quantization test

* fix check dataset

* modify dockerfile

* protect trainer

* style

* test for config

* add more log

* overwrite torch_dtype

* draft doc

* modify quantization_config docstring

* fix class name in docstring

* Apply suggestions from code review

Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>

* more warning

* fix 8bit kwargs tests

* peft compatibility

* remove var

* fix is_gptq_quantized

* remove is_gptq_quantized

* fix wrap

* Update src/transformers/modeling_utils.py

Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>

* add exllama

* skip test

* overwrite float16

* style

* fix skip test

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* fix docsting formatting

* add doc

* better test

---------

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
 2023-08-10 16:06:29 -04:00
Alan Ji
347001237a
docs: add LLaMA-Efficient-Tuning to awesome-transformers (#25441) 
Co-authored-by: statelesshz <jihuazhong1@huawei.com>
 2023-08-10 17:13:39 +02:00
Zach Mueller
a7da2996a0
Fix issue with ratio evaluation steps and auto find batch size (#25436) 
* Fully rebased solution

* 500
 2023-08-10 11:07:32 -04:00
Yih-Dar
2d6839eaa6
Add examples to tests to run when setup.py is modified (#25437) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-08-10 16:42:05 +02:00
Merve Noyan
e7b001db4f
Fix rendering for torch.compile() docs (#25432) 
fix rendering
 2023-08-10 13:25:00 +02:00
Joao Gante
3e41cf13fc
Generate: Load generation config when device_map is passed (#25413) 2023-08-10 10:54:26 +01:00
Sanchit Gandhi
d0839f1a74
[WavLM] Fix Arxiv link and authors (#25415) 
* [WavLM] Fix Arxiv link and authors

* make style
 2023-08-10 10:50:12 +01:00
Joao Gante
123ad5363f
Generation: strict generation config validation at save time (#25411) 
* strict gen config save; Add tests

* add note that the warning will be an exception in v4.34
 2023-08-10 10:42:34 +01:00
Sylvain Gugger
16edf4d9fd
Doc checks (#25408) 
* Document check_dummies

* Type hints and doc in other files

* Document check inits

* Add documentation to

* Address review comments
 2023-08-10 10:53:22 +02:00
TaeYupNoh
b14d4641f6
🌐 [i18n-KO] Translated philosophy.md to Korean (#25010) 
* docs: ko: philosophy.md

* feat: chatgpt draft

* fix: manual edits

* fix: resolve suggestions
 2023-08-10 09:50:51 +02:00
NielsRogge
b175fc39d9
[DINOv2] Update pooler output (#25392) 
Update pooler output
 2023-08-10 09:13:52 +02:00
Joao Gante
d0c1aebea4
Bark: flexible generation config overload (#25414) 2023-08-09 18:51:51 +01:00
amyeroberts
944ddce8bf
Enable passing number of channels when inferring data format (#25412) 2023-08-09 17:41:21 +01:00
hukuda222
cb3c821cb7
aligned sample_beam output selection with beam_search (#25375) 
* aligned sample_beam specs with beam_search

* pull origin main

* Revert "pull origin main"

This reverts commit 06d356f113.

* update test_utils.py

* fix format

* remove comment

---------

Co-authored-by: Shogo Fujita <shogo.fujita@legalontech.jp>
 2023-08-09 18:28:57 +02:00
Yoach Lacombe
704bf595eb
Update Bark generation configs and tests (#25409) 
* update bark generation configs for more coherent parameter

* make style

* update bark hub repo
 2023-08-09 18:28:02 +02:00
Hyeonseo Yun
cf84738d2e
🌐 [i18n-KO] Translated model_summary.md to Korean (#24625) 
* docs: ko: model_summary.md

* feat: nmt and manual edit model_summary.mdx

* fix: resolve suggestions

Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>

* fix: resolve suggestions2

Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>

---------

Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
 2023-08-09 18:27:27 +02:00
MinJae Kang
133aac09b0
🌐 [i18n-KO] Translated add_new_model.md to Korean (#24957) 
* docs: ko: add_new_model.md

* feat: chatgpt draft

* fix: manual edits

* fix: change document title

* fix: edit with reviewers

Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>

* fix: edit with reviewers

Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>

* fix: edit with reviewers

Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>

* fix: edit with reviewers

Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>

* fix: edit with reviewers

Co-authored-by: SeongWooChoi <46990061+nuatmochoi@users.noreply.github.com>

* fix: edit with reviewers

Co-authored-by: SeongWooChoi <46990061+nuatmochoi@users.noreply.github.com>

* fix: edit with reviewers

Co-authored-by: SeongWooChoi <46990061+nuatmochoi@users.noreply.github.com>

* fix: edit with reviewers

Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>

* fix: add anchor to header

* Update docs/source/ko/add_new_model.md

Co-authored-by: 이서정 <97655267+sjlee-wise@users.noreply.github.com>

* Update docs/source/ko/add_new_model.md

Co-authored-by: 이서정 <97655267+sjlee-wise@users.noreply.github.com>

* Update docs/source/ko/add_new_model.md

Co-authored-by: 이서정 <97655267+sjlee-wise@users.noreply.github.com>

* fix: edit with reviews

* feat: edit toctree

---------

Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>
Co-authored-by: SeongWooChoi <46990061+nuatmochoi@users.noreply.github.com>
Co-authored-by: 이서정 <97655267+sjlee-wise@users.noreply.github.com>
 2023-08-09 18:24:29 +02:00
Maria Khalusova
f2a43c7383
VQA task guide (#25244) 
* initial commit

* semi-finished task guide draft

* image link

* Apply suggestions from code review

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/tasks/visual_question_answering.md

Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>

* feedback addressed

* Apply suggestions from code review

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* nits addressed

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2023-08-09 08:29:06 -04:00
Joao Gante
eb3ded16f7
Generate: lower severity of parameterization checks (#25407) 2023-08-09 13:15:06 +01:00
David Reguera
ef74da6582
16059 - Add extra type hints for AltCLIPModel (#25399) 2023-08-09 13:13:33 +01:00
Joao Gante
f456b4d10b
Generate: generation config validation fixes in docs (#25405) 2023-08-09 13:07:11 +01:00
Alan Ji
00b93cda21
Improve training args (#25401) 
* enhanced tips for some training args

* make style
 2023-08-09 13:50:13 +02:00
Joao Gante
3deed1f97e
Generate: length validation (#25384) 2023-08-09 11:48:32 +01:00
Joao Gante
d59b872c9e
Docs: introduction to generation with LLMs (#25240) 
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2023-08-09 11:09:20 +01:00
amyeroberts
ea5dda2290
YOLOS - Revert default return_pixel_mask value (#25404) 
Revert default return_pixel_mask value
 2023-08-09 11:09:09 +01:00
Sylvain Gugger
599377161b
Fix path for dynamic module creation (#25402) 2023-08-09 10:46:05 +02:00
jiqing-feng
85447bb22e
rm useless condition since the previous condition contains it. (#25403) 2023-08-09 09:31:24 +02:00
David Reguera
1564a81ac5
16059 - Add missing type hints for ASTModel (#25364) 
* 16059 - Add missing type hints for ASTModel

* Add an additional type hint

Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>

---------

Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>
 2023-08-09 08:31:57 +02:00
SeongWooChoi
1367142afd
🌐 [i18n-KO] Translated perf_train_cpu_many.md to Korean (#24923) 
* docs: ko: perf_train_cpu_many.md

* feat: chatgpt draft

* fix: manual edits

* fix: resolve suggestions

Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>

---------

Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>
 2023-08-09 08:15:31 +02:00
Abhipsha Das
41c5f45bfe
[DOCS] Add example for TopPLogitsWarper (#25361) 
* [DOCS] Add example for `TopPLogitsWarper`

* fix typo

* address review feedback

* address review nits
 2023-08-08 19:18:33 +02:00
Marc Sun
3a05e010e0
change version (#25387) 2023-08-08 13:05:41 -04:00
amyeroberts
e3490104da
Add copied from for image processor methods (#25121) 
* Add copied from statements for image processors

* Move out rescale and normalize to base image processor

* Remove rescale and normalize from vit (post rebase)

* Update docstrings and tidy up

* PR comments
 2023-08-08 17:02:49 +01:00
Yih-Dar
5b517e1764
Use small config for OneFormerModelTest.test_model_with_labels (#25383) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-08-08 17:15:34 +02:00
Yih-Dar
9c7b744795
Fix missing usage of token (#25382) 
* add missing tokens

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-08-08 16:27:24 +02:00
Joao Gante
5bd8c011bb
Generate: add config-level validation (#25381) 2023-08-08 13:53:03 +01:00
Yih-Dar
9e57e0c063
Fix torch_job worker(s) crashing (#25374) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-08-08 14:12:56 +02:00
나건주
6247d1b2b6
🌐 [i18n-KO] Translated add_tensorflow_model.md to Korean (#25017) 
* docs: ko: add_tensorflow_model.md

* feat: chatgpt draft

* fix: manual edits

* fix: manual edits

* fix: resolve suggestions

* fix: manual edits
 2023-08-08 13:56:34 +02:00
Alan Ji
26ce4dd8b7
Enable tests to run on third-party devcies (#25327) 
* enable unit tests to run on third-party devcies other than CUDA and CPU.

* remove the modification that enabled ut on MPS

* control test on third-party device by env variable

* update

---------

Co-authored-by: statelesshz <jihuazhong1@huawei.com>
 2023-08-08 13:48:50 +02:00
Yih-Dar
5744482abc
Fix token in example template (#25351) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-08-08 12:00:31 +02:00
Zach Mueller
01ab39b65f
Load state in else (#25318) 
* Load else

* New approach

* Propagate
 2023-08-08 05:41:00 -04:00
amyeroberts
36d5b8b06c
MaskFormer, Mask2Former - replace einsum for tracing (#25297) 
* Replace einsum with ops for tracing

* Fix comment
 2023-08-08 10:37:14 +01:00
Sanchit Gandhi
dedd11160d
[ASR Pipeline] Clarify return timestamps (#25344) 
* [ASR Pipeline] Clarify return timestamps

* fix indentation

* fix ctc check

* fix ctc error message!

* fix test

* fix other test

* add new tests

* final comment
 2023-08-08 10:16:00 +01:00
JB (Don)
5ea2595ecd
Add warning for missing attention mask when pad tokens are detected (#25345) 
* Add attention mask and pad token warning to many of the models

* Remove changes under examples/research_projects

These files are not maintained by HG.

* Skip the warning check during torch.fx or JIT tracing

* Switch ordering for the warning and input shape assignment

This ordering is a little cleaner for some of the cases.

* Add missing line break in one of the files
 2023-08-08 10:49:21 +02:00