riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-31 02:02:21 +06:00
Code Issues Actions 33 Packages Projects Releases Wiki Activity
13,614 Commits 887 Branches 232 Tags 2.1 GiB
b0f23036f1
Commit Graph

13614 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Yih-Dar
b0f23036f1
Update TF pin in docker image (#25343) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-08-07 12:32:34 +02:00
Injin Paek
b9da44bd3e
🌐 [i18n-KO] Translated perf_infer_gpu_one.md to Korean (#24978) 
* docs: ko: perf_infer_gpu_one

* feat: chatgpt draft

* fix: manual edits

* fix: manual edits

* fix: resolve suggestions

Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-authored-by: TaeYupNoh <107118671+TaeYupNoh@users.noreply.github.com>

* fix: resolve suggestions

* fix: resolve suggestions

Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>

---------

Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-authored-by: TaeYupNoh <107118671+TaeYupNoh@users.noreply.github.com>
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
 2023-08-07 08:37:29 +02:00
Guillaume "Vermeille" Sanchez
d533465150
add CFG for .generate() (#24654) 2023-08-06 20:15:24 +01:00
mariecwhite
a6e6b1c622
Remove jnp.DeviceArray since it is deprecated. (#24875) 
* Remove jnp.DeviceArray since it is deprecated.

* Replace all instances of jnp.DeviceArray with jax.Array

* Update src/transformers/models/bert/modeling_flax_bert.py

---------

Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
 2023-08-04 18:36:57 +01:00
Sanchit Gandhi
fdd81aea12
[Whisper] Better error message for outdated generation config (#25298) 2023-08-04 15:53:57 +01:00
Sylvain Gugger
fdaef3368b
Document toc check and doctest check scripts (#25319) 
* Clean doc toc check and make doctest list better

* Add to Makefile
 2023-08-04 16:24:04 +02:00
Yih-Dar
ce6d153a53
Make bark could have tiny model (#25290) 
* temp

* update

* update

* update

* small dim

* small dim

* small dim

* fix

* update

* fix

* fix

* fix

* fix

* fix

* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-08-04 15:13:14 +02:00
Sylvain Gugger
f0fd73a2de
Document check copies (#25291) 
* Document check copies better and add tests

* Include header in check for copies

* Manual fixes

* Try autofix

* Fixes

* Clean tests

* Finalize doc

* Remove debug print

* More fixes
 2023-08-04 14:56:29 +02:00
Sylvain Gugger
29f04002e6
Deal with nested configs better in base class (#25237) 
* Deal better with nested configs

* Fixes

* More fixes

* Fix last test

* Clean up existing configs

* Remove hack in MPT Config

* Update src/transformers/configuration_utils.py

Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>

* Fix setting a nested config via dict in the kwargs

* Adapt common test

* Add test for nested config load with dict

---------

Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
 2023-08-04 14:56:09 +02:00
Sylvain Gugger
aeb5a08abd
Add offline mode for agents (#25226) 
* Add offline mode for agents

* Disable second check too
 2023-08-04 14:55:58 +02:00
Joao Gante
bff4313b37
Generate: get generation mode as an enum (#25292) 2023-08-04 13:35:10 +01:00
Sylvain Gugger
fab1a0aa82
Give more memory in test_disk_offload (#25315) 2023-08-04 14:10:31 +02:00
Peter Law
67683095a6
Move usage of deprecated logging.warn to logging.warning (#25310) 
The former spelling is deprecated and has been discouraged for a
while. The latter spelling seems to be more common in this project
anyway, so this change ought to be safe.

Fixes https://github.com/huggingface/transformers/issues/25283
 2023-08-04 12:42:05 +01:00
Victor Geislinger
641adca558
Fix typo: Roberta -> RoBERTa (#25302) 2023-08-03 14:17:30 -07:00
Howard Huang
33da2db5ea
[small] llama2.md typo (#25295) 
`groupe` -> `grouped`
 2023-08-03 14:17:06 -07:00
Sanchit Gandhi
66c240f3c9
[JAX] Bump min version (#25286) 
* [JAX] Bump min version

* make fixup
 2023-08-03 16:05:02 +01:00
Roland Szabo
d114a6b71f
Add timeout parameter to load_image function (#25184) 
* Add timeout parameter to load_image function.

* Remove line.

* Reformat code

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* Add parameter to docs.

---------

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2023-08-03 15:51:54 +01:00
Yoach Lacombe
6d3f9c1e2e
add generate method to SpeechT5ForTextToSpeech (#25233) 
* add generate method to SpeechT5ForTextToSpeech

* update speecht5forTTS docstrings

* Remove defaults to None in generate docstrings

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

---------

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2023-08-03 14:12:07 +01:00
Yoach Lacombe
8455346c5c
Update bark doc (#25234) 
* add mention to optimization in Bark docs

* add offload mention in docs

* Apply suggestions from code review

Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>

* Update bark docs.

* Update bark.md

---------

Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
 2023-08-03 14:08:39 +01:00
Joao Gante
a8817371c9
Docs: separate generate section (#25235) 
Separate generate doc section
 2023-08-03 13:51:56 +01:00
amyeroberts
30409af6e1
Update InstructBLIP & Align values after rescale update (#25209) 
* Update InstructBLIP values
Note: the tests are not independent. Running the test independentely produces different logits compared to running all the integration tests

* Update test values after rescale update

* Remove left over commented out code

* Revert to previous rescaling logic

* Update rescale tests
 2023-08-03 11:01:10 +01:00
Tom Aarsen
15082a9dc6
Docs: Update list of report_to logging integrations in docstring (#25281) 
* Update list of logging integrations in docstring

Also update type hint

* Also add 'flyte' to report_to callback list

* Revert 'report_to' type hint update

Due to CLI breaking
 2023-08-03 11:34:45 +02:00
Yih-Dar
2bd7a27a67
CI with pytest_num_workers=8 for torch/tf jobs (#25274) 
n8

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-08-02 22:00:32 +02:00
Yih-Dar
bd90cda9a6
CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 
* CI with layers=2

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-08-02 20:22:36 +02:00
Patrick von Platen
b28ebb2655
[MMS] Fix mms (#25267) 
* [MMS] Fix mms

* [MMS] Fix mms

* fix mms loading

* Apply suggestions from code review

* make style

* Update tests/models/wav2vec2/test_modeling_wav2vec2.py
 2023-08-02 18:11:15 +02:00
Kevin Lloyd Bernal
ad8321512d
recommend DeepSpeed's Argument Parsing documentation (#25268) 2023-08-02 11:48:39 -04:00
heuristicwave
bef02fd6b9
🌐 [i18n-KO] Translated perf_infer_gpu_many.md to Korean (#24943) 
* doc: ko: perf_infer_gpu_many.mdx

* feat: chatgpt draft

* fix: manual edits

* Update docs/source/ko/perf_infer_gpu_many.md

Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>

---------

Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>
 2023-08-02 16:06:35 +02:00
Yih-Dar
8edd0da960
Remove pytest_options={"rA": None} in CI (#25263) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-08-02 14:53:05 +02:00
Euan Ong
1baeed5bdf
Fix return_dict_in_generate bug in InstructBlip generate function (#25246) 
Fix bug in InstructBlip generate function

Previously, the postprocessing conducted on generated sequences in InstructBlip's generate function assumed these sequences were tensors (i.e. that `return_dict_in_generate == False`).

This commit checks whether the result of the call to the wrapped language model `generate()` is a tensor, and if not attempts to postprocess the sequence attribute of the returned results object.
 2023-08-02 13:43:54 +01:00
Ashish Thomas Chempolil
eec0d84e6a
[DOCS] Add example and modified docs of EtaLogitsWarper (#25125) 
* added example and modified docs for EtaLogitsWarper

* make style

* fixed styling issue on 544

* removed error info and added set_seed

* Update src/transformers/generation/logits_process.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* Update src/transformers/generation/logits_process.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* updated the results

---------

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2023-08-02 11:55:56 +01:00
Yupeng Jia
8021c684ec
Fix some bugs for two stage training of deformable detr (#25045) 
* Update modeling_deformable_detr.py

Fix bugs for two stage training

* Update modeling_deformable_detr.py

* Add test_two_stage_training to DeformableDetrModelTest

---------

Co-authored-by: yupeng.jia <yupeng.jia@momenta.ai>
 2023-08-02 11:30:36 +01:00
amyeroberts
1b35409768
Update rescale tests - cast to float after rescaling to reflect #25229 (#25259) 
Rescale tests - cast to float after rescaling to reflect #25229
 2023-08-02 11:29:55 +01:00
Sourab Mangrulkar
904e7e0f3c
resolving zero3 init when using accelerate config with Trainer (#25227) 
* resolving zero3 init when using accelerate config with Trainer

* refactor

* fix

* fix import
 2023-08-02 15:07:27 +05:30
Yih-Dar
149cb0cce2
Add token arugment in example scripts (#25172) 
* fix

* fix

* fix

* fix

* fix

* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-08-02 11:17:31 +02:00
YQ
c6a8768dab
add pathname and line number to logging formatter in debug mode (#25203) 
* add pathname and lineno to logging formatter in debug mode

* use TRANSFORMERS_VERBOSITY="detail" to print pathname and lineno
 2023-08-02 09:44:43 +01:00
YQ
2230d149f0
fix get_keys_to_not_convert() to return correct modules for full precision inference (#25105) 
* add test for `get_keys_to_not_convert`

* add minimum patch to keep mpt lm_head from 8bit quantization

* add reivsion to
 2023-08-02 04:21:52 -04:00
Sylvain Gugger
f6f567d0be
Fix set of model parallel in the Trainer when no GPUs are available (#25239) 2023-08-02 03:29:00 -04:00
amyeroberts
d27e4c18fe
Move rescale dtype recasting to match torchvision ToTensor (#25229) 
Move dtype recasting to match torchvision ToTensor
 2023-08-01 12:33:12 +01:00
Younes Belkada
3170af71e1
[Detr] Fix detr BatchNorm replacement issue (#25230) 
* fix detr weird issue

* Update src/transformers/models/conditional_detr/modeling_conditional_detr.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* fix copies

* fix copies

---------

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2023-08-01 12:21:48 +02:00
Younes Belkada
05ebb0264e
[MPT] Add require_bitsandbytes on MPT integration tests (#25201) 
* add  `require_bitsandbytes` on MPT integration tests

* add it on mpt as well
 2023-08-01 12:20:34 +02:00
Younes Belkada
972fdcc778
[Docs/quantization] Clearer explanation on how things works under the hood. + remove outdated info (#25216) 
* clearer explanation on how things works under the hood.

* Update docs/source/en/main_classes/quantization.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/main_classes/quantization.md

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* add `load_in_4bit` in `from_pretrained`

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2023-08-01 10:56:52 +02:00
Younes Belkada
77c3973e8f
[Pix2Struct] Fix pix2struct cross attention (#25200) 
* fix pix2struct cross attention

* fix torchscript slow test
 2023-08-01 10:56:37 +02:00
Wang, Yi
4033ea7167
make build_mpt_alibi_tensor a method of MptModel so that deepspeed co… (#25193) 
make build_mpt_alibi_tensor a method of MptModel so that deepspeed could override it to make autoTP work

Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
 2023-08-01 01:35:49 -04:00
Yih-Dar
0fd8d2aa2c
Fix docker image build failure (#25214) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-07-31 20:13:15 +02:00
Yih-Dar
1b4f6199c6
Update tiny model info. and pipeline testing (#25213) 
* update tiny_model_summary.json

* update

* update

* update

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-07-31 19:35:33 +02:00
Younes Belkada
e0c50b274a
[pipeline] revisit device check for pipeline (#25207) 
* revisit device check for pipeline

* let's raise an error.
 2023-07-31 18:43:21 +02:00
Stas Bekman
5220606607
[quantization.md] fix (#25190) 
Update quantization.md
 2023-07-31 09:37:29 -07:00
Yih-Dar
9ca3aa0156
Fix all_model_classes in FlaxBloomGenerationTest (#25211) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-07-31 17:32:05 +02:00
Younes Belkada
59dcea3fe4
[PreTrainedModel] Wrap cuda and to method correctly (#25206) 
wrap `cuda` and `to` method correctly
 2023-07-31 17:25:09 +02:00
Yih-Dar
67b85f24de
Better error message in _prepare_output_docstrings (#25202) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-07-31 16:15:02 +02:00