riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-14 18:18:24 +06:00
Code Issues Actions 15 Packages Projects Releases Wiki Activity
15,053 Commits 864 Branches 226 Tags 2 GiB
3f9f749325
Commit Graph

15053 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Sylvain Gugger
fab1a0aa82
Give more memory in test_disk_offload (#25315) 2023-08-04 14:10:31 +02:00
Peter Law
67683095a6
Move usage of deprecated logging.warn to logging.warning (#25310) 
The former spelling is deprecated and has been discouraged for a
while. The latter spelling seems to be more common in this project
anyway, so this change ought to be safe.

Fixes https://github.com/huggingface/transformers/issues/25283
 2023-08-04 12:42:05 +01:00
Victor Geislinger
641adca558
Fix typo: Roberta -> RoBERTa (#25302) 2023-08-03 14:17:30 -07:00
Howard Huang
33da2db5ea
[small] llama2.md typo (#25295) 
`groupe` -> `grouped`
 2023-08-03 14:17:06 -07:00
Sanchit Gandhi
66c240f3c9
[JAX] Bump min version (#25286) 
* [JAX] Bump min version

* make fixup
 2023-08-03 16:05:02 +01:00
Roland Szabo
d114a6b71f
Add timeout parameter to load_image function (#25184) 
* Add timeout parameter to load_image function.

* Remove line.

* Reformat code

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* Add parameter to docs.

---------

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2023-08-03 15:51:54 +01:00
Yoach Lacombe
6d3f9c1e2e
add generate method to SpeechT5ForTextToSpeech (#25233) 
* add generate method to SpeechT5ForTextToSpeech

* update speecht5forTTS docstrings

* Remove defaults to None in generate docstrings

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

---------

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2023-08-03 14:12:07 +01:00
Yoach Lacombe
8455346c5c
Update bark doc (#25234) 
* add mention to optimization in Bark docs

* add offload mention in docs

* Apply suggestions from code review

Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>

* Update bark docs.

* Update bark.md

---------

Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
 2023-08-03 14:08:39 +01:00
Joao Gante
a8817371c9
Docs: separate generate section (#25235) 
Separate generate doc section
 2023-08-03 13:51:56 +01:00
amyeroberts
30409af6e1
Update InstructBLIP & Align values after rescale update (#25209) 
* Update InstructBLIP values
Note: the tests are not independent. Running the test independentely produces different logits compared to running all the integration tests

* Update test values after rescale update

* Remove left over commented out code

* Revert to previous rescaling logic

* Update rescale tests
 2023-08-03 11:01:10 +01:00
Tom Aarsen
15082a9dc6
Docs: Update list of report_to logging integrations in docstring (#25281) 
* Update list of logging integrations in docstring

Also update type hint

* Also add 'flyte' to report_to callback list

* Revert 'report_to' type hint update

Due to CLI breaking
 2023-08-03 11:34:45 +02:00
Yih-Dar
2bd7a27a67
CI with pytest_num_workers=8 for torch/tf jobs (#25274) 
n8

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-08-02 22:00:32 +02:00
Yih-Dar
bd90cda9a6
CI with num_hidden_layers=2 🚀🚀🚀 (#25266) 
* CI with layers=2

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-08-02 20:22:36 +02:00
Patrick von Platen
b28ebb2655
[MMS] Fix mms (#25267) 
* [MMS] Fix mms

* [MMS] Fix mms

* fix mms loading

* Apply suggestions from code review

* make style

* Update tests/models/wav2vec2/test_modeling_wav2vec2.py
 2023-08-02 18:11:15 +02:00
Kevin Lloyd Bernal
ad8321512d
recommend DeepSpeed's Argument Parsing documentation (#25268) 2023-08-02 11:48:39 -04:00
heuristicwave
bef02fd6b9
🌐 [i18n-KO] Translated perf_infer_gpu_many.md to Korean (#24943) 
* doc: ko: perf_infer_gpu_many.mdx

* feat: chatgpt draft

* fix: manual edits

* Update docs/source/ko/perf_infer_gpu_many.md

Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>

---------

Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>
 2023-08-02 16:06:35 +02:00
Yih-Dar
8edd0da960
Remove pytest_options={"rA": None} in CI (#25263) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-08-02 14:53:05 +02:00
Euan Ong
1baeed5bdf
Fix return_dict_in_generate bug in InstructBlip generate function (#25246) 
Fix bug in InstructBlip generate function

Previously, the postprocessing conducted on generated sequences in InstructBlip's generate function assumed these sequences were tensors (i.e. that `return_dict_in_generate == False`).

This commit checks whether the result of the call to the wrapped language model `generate()` is a tensor, and if not attempts to postprocess the sequence attribute of the returned results object.
 2023-08-02 13:43:54 +01:00
Ashish Thomas Chempolil
eec0d84e6a
[DOCS] Add example and modified docs of EtaLogitsWarper (#25125) 
* added example and modified docs for EtaLogitsWarper

* make style

* fixed styling issue on 544

* removed error info and added set_seed

* Update src/transformers/generation/logits_process.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* Update src/transformers/generation/logits_process.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* updated the results

---------

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2023-08-02 11:55:56 +01:00
Yupeng Jia
8021c684ec
Fix some bugs for two stage training of deformable detr (#25045) 
* Update modeling_deformable_detr.py

Fix bugs for two stage training

* Update modeling_deformable_detr.py

* Add test_two_stage_training to DeformableDetrModelTest

---------

Co-authored-by: yupeng.jia <yupeng.jia@momenta.ai>
 2023-08-02 11:30:36 +01:00
amyeroberts
1b35409768
Update rescale tests - cast to float after rescaling to reflect #25229 (#25259) 
Rescale tests - cast to float after rescaling to reflect #25229
 2023-08-02 11:29:55 +01:00
Sourab Mangrulkar
904e7e0f3c
resolving zero3 init when using accelerate config with Trainer (#25227) 
* resolving zero3 init when using accelerate config with Trainer

* refactor

* fix

* fix import
 2023-08-02 15:07:27 +05:30
Yih-Dar
149cb0cce2
Add token arugment in example scripts (#25172) 
* fix

* fix

* fix

* fix

* fix

* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-08-02 11:17:31 +02:00
YQ
c6a8768dab
add pathname and line number to logging formatter in debug mode (#25203) 
* add pathname and lineno to logging formatter in debug mode

* use TRANSFORMERS_VERBOSITY="detail" to print pathname and lineno
 2023-08-02 09:44:43 +01:00
YQ
2230d149f0
fix get_keys_to_not_convert() to return correct modules for full precision inference (#25105) 
* add test for `get_keys_to_not_convert`

* add minimum patch to keep mpt lm_head from 8bit quantization

* add reivsion to
 2023-08-02 04:21:52 -04:00
Sylvain Gugger
f6f567d0be
Fix set of model parallel in the Trainer when no GPUs are available (#25239) 2023-08-02 03:29:00 -04:00
amyeroberts
d27e4c18fe
Move rescale dtype recasting to match torchvision ToTensor (#25229) 
Move dtype recasting to match torchvision ToTensor
 2023-08-01 12:33:12 +01:00
Younes Belkada
3170af71e1
[Detr] Fix detr BatchNorm replacement issue (#25230) 
* fix detr weird issue

* Update src/transformers/models/conditional_detr/modeling_conditional_detr.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* fix copies

* fix copies

---------

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2023-08-01 12:21:48 +02:00
Younes Belkada
05ebb0264e
[MPT] Add require_bitsandbytes on MPT integration tests (#25201) 
* add  `require_bitsandbytes` on MPT integration tests

* add it on mpt as well
 2023-08-01 12:20:34 +02:00
Younes Belkada
972fdcc778
[Docs/quantization] Clearer explanation on how things works under the hood. + remove outdated info (#25216) 
* clearer explanation on how things works under the hood.

* Update docs/source/en/main_classes/quantization.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/main_classes/quantization.md

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* add `load_in_4bit` in `from_pretrained`

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2023-08-01 10:56:52 +02:00
Younes Belkada
77c3973e8f
[Pix2Struct] Fix pix2struct cross attention (#25200) 
* fix pix2struct cross attention

* fix torchscript slow test
 2023-08-01 10:56:37 +02:00
Wang, Yi
4033ea7167
make build_mpt_alibi_tensor a method of MptModel so that deepspeed co… (#25193) 
make build_mpt_alibi_tensor a method of MptModel so that deepspeed could override it to make autoTP work

Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
 2023-08-01 01:35:49 -04:00
Yih-Dar
0fd8d2aa2c
Fix docker image build failure (#25214) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-07-31 20:13:15 +02:00
Yih-Dar
1b4f6199c6
Update tiny model info. and pipeline testing (#25213) 
* update tiny_model_summary.json

* update

* update

* update

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-07-31 19:35:33 +02:00
Younes Belkada
e0c50b274a
[pipeline] revisit device check for pipeline (#25207) 
* revisit device check for pipeline

* let's raise an error.
 2023-07-31 18:43:21 +02:00
Stas Bekman
5220606607
[quantization.md] fix (#25190) 
Update quantization.md
 2023-07-31 09:37:29 -07:00
Yih-Dar
9ca3aa0156
Fix all_model_classes in FlaxBloomGenerationTest (#25211) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-07-31 17:32:05 +02:00
Younes Belkada
59dcea3fe4
[PreTrainedModel] Wrap cuda and to method correctly (#25206) 
wrap `cuda` and `to` method correctly
 2023-07-31 17:25:09 +02:00
Yih-Dar
67b85f24de
Better error message in _prepare_output_docstrings (#25202) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-07-31 16:15:02 +02:00
Joao Gante
4a564490e1
Musicgen: CFG is manually added (#25173) 2023-07-31 11:21:11 +01:00
amyeroberts
05cda5df34
🚨🚨🚨 Fix rescale ViVit Efficientnet (#25174) 
* Fix rescaling bug

* Add tests

* Update integration tests

* Fix up

* Update src/transformers/image_transforms.py

* Update test - new possible order in list
 2023-07-28 19:52:51 +01:00
Sanchit Gandhi
03f98f9683
[MusicGen] Fix integration tests (#25169) 
* move to device

* update with cuda values

* fix fp16

* more rigorous
 2023-07-28 18:50:15 +01:00
Yoni Gottesman
c90e14fb0f
Fix beam search to sample at least 1 non eos token (#25103) (#25115) 2023-07-28 13:20:24 -04:00
Sohyun Sim
31f137c04f
🌐 [i18n-KO] Translated transformers_agents.md to Korean (#24881) 
* docs: ko: transformers_agents.md

* docs: ko: transformers_agents.md

* feat: deepl draft

* fix: manual edits

* fix: resolve suggestions

Co-authored-by: Juntae <79131091+sronger@users.noreply.github.com>
Co-authored-by: Injin Paek <71638597+eenzeenee@users.noreply.github.com>

---------

Co-authored-by: Juntae <79131091+sronger@users.noreply.github.com>
Co-authored-by: Injin Paek <71638597+eenzeenee@users.noreply.github.com>
 2023-07-28 13:06:37 -04:00
Younes Belkada
dd9d45b6ec
[InstructBlip] Fix instructblip slow test (#25171) 
* fix instruct blip slow test

* Update tests/models/instructblip/test_modeling_instructblip.py
 2023-07-28 17:00:10 +02:00
Younes Belkada
add0895dd9
[Mpt] Fix mpt slow test (#25170) 
fix mpt slow test
 2023-07-28 16:45:09 +02:00
Yih-Dar
d53b8ad780
Update use_auth_token -> token in example scripts (#25167) 
* pytorch examples

* tensorflow examples

* flax examples

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-07-28 15:33:45 +02:00
Alexander Markov
3cbc560d03
added compiled model support for inference (#25124) 
* added compiled model support for inference

* linter

* Fix tests

* linter

* linter

* remove inference mode from pipelines

* Linter

---------

Co-authored-by: amarkov <alexander@inworld.ai>
 2023-07-28 08:28:04 -04:00
Alan Ji
afa96fffdf
make run_generation more generic for other devices (#25133) 
* make run_generation more generic for other devices

* use Accelerate to support any device type it supports.

* make style

* fix error usage of accelerator.prepare_model

* use `PartialState` to make sure everything is running on the right device

---------

Co-authored-by: statelesshz <jihuazhong1@huawei.com>
 2023-07-28 08:20:10 -04:00
jiqing-feng
d23d2c27c2
Represent query_length in a different way to solve jit issue (#25164) 
Fix jit trace
 2023-07-28 08:19:10 -04:00