riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-31 02:02:21 +06:00
Code Issues Actions 23 Packages Projects Releases Wiki Activity
16,022 Commits 887 Branches 232 Tags 2.1 GiB
5c88253556
Commit Graph

16022 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Dhruv Pai
5c88253556
Add on_optimizer_step to callback options (#31095) 
* Modified test

* Added on_optimizer_step to callbacks

* Move callback after step is called

* Added on optimizer step callback
 2024-05-29 16:20:59 +02:00
Joao Gante
4af705c6ce
Add VLM generation default contributor (#31115) 
* add Raushan

* add Raushan
 2024-05-29 15:17:14 +01:00
Younes Belkada
cb879c5801
FIX / Docs: Fix GPTQ expected number of bits (#31111) 
Update overview.md
 2024-05-29 15:56:28 +02:00
Yih-Dar
1f84141391
Fix nightly circleci (#31114) 
* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-05-29 15:42:39 +02:00
Zach Mueller
d16053c867
Rm maintainer + migrate (#31089) 2024-05-29 09:35:37 -04:00
Matt
0bef4a2738
Fix faulty rstrip in module loading (#31108) 2024-05-29 13:33:26 +01:00
Matt
97a58a5d2c
Fix env.py in cases where torch is not present (#31113) 
* Fix env.py in cases where torch is not present

* Simplify the fix (and avoid some issues)
 2024-05-29 13:20:36 +01:00
Huazhong Ji
c8861376ad
Improve transformers-cli env reporting (#31003) 
* Improve `transformers-cli env` reporting

* move the line `"Using GPU in script?": "<fill in>"` to in if conditional
statement

* same option for npu
 2024-05-29 11:57:54 +01:00
Lucain
c3044ec2f3
Use HF_HUB_OFFLINE + fix has_file in offline mode (#31016) 
* Fix has_file in offline mode

* harmonize env variable for offline mode

* Switch to HF_HUB_OFFLINE

* fix test

* revert test_offline to test TRANSFORMERS_OFFLINE

* Add new offline test

* merge conflicts

* docs
 2024-05-29 11:55:43 +01:00
Younes Belkada
bfe6f513b9
FEAT: Add mistral v3 conversion script (#30981) 
* add mistral v3 conversion script

* Update src/transformers/models/mistral/convert_mistral_weights_to_hf.py

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* fixup

---------

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
 2024-05-29 11:43:54 +02:00
Raushan Turganbay
d521ba5797
Quantized KV cache: update quanto (#31052) 
* quanto latest version was refactored

* add error msg

* incorrect compare sign

* Update src/transformers/cache_utils.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

---------

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2024-05-29 14:25:44 +05:00
amyeroberts
a564d10afe
Deprecate low use models (#30781) 
* Deprecate models
- graphormer
- time_series_transformer
- xlm_prophetnet
- qdqbert
- nat
- ernie_m
- tvlt
- nezha
- mega
- jukebox
- vit_hybrid
- x_clip
- deta
- speech_to_text_2
- efficientformer
- realm
- gptsan_japanese

* Fix up

* Fix speech2text2 imports

* Make sure message isn't indented

* Fix docstrings

* Correctly map for deprecated models from model_type

* Uncomment out

* Add back time series transformer and x-clip

* Import fix and fix-up

* Fix up with updated ruff
 2024-05-28 18:07:07 +01:00
Younes Belkada
7f08817be4
Docs / Quantization: Redirect deleted page (#31063) 
Update _redirects.yml
 2024-05-28 18:29:22 +02:00
Younes Belkada
3264be4114
TST: Fix instruct-blip tests (#31088) 
* fix flan t5 tests

* better format
 2024-05-28 18:29:11 +02:00
Jonny Li
476890e9ae
Fix DeepSpeed compatibility with weight_norm (#30881) (#31018) 2024-05-28 17:25:15 +01:00
Albert Villanova del Moral
aada568f73
Fix PretrainedConfig docstring with deprecated resume_download (#31014) 2024-05-28 17:47:35 +02:00
Yih-Dar
3af7bf30ad
skip test_multi_gpu_data_parallel_forward for vit and deit (#31086) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-05-28 17:44:52 +02:00
Younes Belkada
ab19f907fd
FIX / OPT: Fix OPT multi-GPU training for OPTForQuestionAnswering (#31092) 
Update modeling_opt.py
 2024-05-28 17:06:00 +02:00
Younes Belkada
94d416f018
FIX: Add accelerate as a hard requirement (#31090) 
add accelerate
 2024-05-28 17:05:44 +02:00
Sigbjørn Skjæret
22dab246c5
Render chat template tojson filter as unicode (#31041) 
* Render chat template tojson filter as unicode

* ruff--
 2024-05-28 15:02:51 +01:00
Younes Belkada
4f98b14465
Docs / PEFT: Add PEFT API documentation (#31078) 
* add peft references

* add peft references

* Update docs/source/en/peft.md

* Update docs/source/en/peft.md
 2024-05-28 15:04:43 +02:00
Raushan Turganbay
779bc360ff
Watermark: fix tests (#30961) 
* fix tests

* style

* Update tests/generation/test_utils.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

---------

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2024-05-28 17:07:42 +05:00
Lysandre Debut
a3c7b59e31
Fix failing tokenizer tests (#31083) 
* Fix failing tokenizer tests

* Use small tokenizer

* Fix remaining reference
 2024-05-28 13:34:23 +02:00
NielsRogge
90da0b1c9f
[SuperPoint, PaliGemma] Update docs (#31025) 
* Update docs

* Add PaliGemma resources

* Address comment

* Update docs
 2024-05-28 13:22:06 +02:00
Sina Taslimi
66add161dc
Fix typo in trainer.py (#31048) 2024-05-28 12:09:32 +01:00
Pavel Iakubovskii
98e2d48e9a
Fix OWLv2 post_process_object_detection for multiple images (#31082) 
* Add test for multiple images

* [run slow] owlv2

* Fix box rescaling

* [run slow] owlv2
 2024-05-28 12:06:06 +01:00
Pavel Iakubovskii
c31473ed44
Remove float64 cast for OwlVit and OwlV2 to support MPS device (#31071) 
Remove float64
 2024-05-28 11:41:40 +01:00
oOraph
936ab7bae5
fix from_pretrained in offline mode when model is preloaded in cache (#31010) 
* Unit test to verify fix

Signed-off-by: Raphael Glon <oOraph@users.noreply.github.com>

* fix from_pretrained in offline mode when model is preloaded in cache

Signed-off-by: Raphael Glon <oOraph@users.noreply.github.com>

* minor: fmt

Signed-off-by: Raphael Glon <oOraph@users.noreply.github.com>

---------

Signed-off-by: Raphael Glon <oOraph@users.noreply.github.com>
Co-authored-by: Raphael Glon <oOraph@users.noreply.github.com>
 2024-05-28 11:56:05 +02:00
Hengwen Tong
537deb7869
Remove redundant backend checks in training_args.py (#30999) 
* Remove backend checks in training_args.py

* Expilicit initialize the device

---------

Co-authored-by: tonghengwen <tonghengwen@cambricon.com>
 2024-05-28 11:52:47 +02:00
AP
dd4654eab7
Update quicktour.md to fix broken link to Glossary (#31072) 
Update quicktour.md to fix broken link

Missing '/' in attention mask link in the transformers quicktour
 2024-05-28 11:50:45 +02:00
Clint Adams
e18da4e3f2
fix "piano" typo (#31027) 2024-05-28 11:48:23 +02:00
Yih-Dar
8e3b1fef97
Remove ninja from docker image build (#31080) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-05-28 11:36:26 +02:00
Yih-Dar
8f0f7271d0
use @main (#31065) 
use main

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-05-28 10:53:28 +02:00
Yih-Dar
9d35edbb30
skip test_model_parallelism for 2 model test classes (#31067) 
skip

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-05-27 18:36:39 +02:00
Yoach Lacombe
d355741eca
Fix pad_to_max_length Whisper (#30787) 
* fix pad_to_max_length Whisper

* add tests

* make style
 2024-05-27 16:09:05 +02:00
Marc Sun
b84cd67526
Fix quanto tests (#31062) 
fix quanto tests
 2024-05-27 15:53:45 +02:00
amyeroberts
cd797778e4
Update feature request label in template (#30940) 2024-05-27 15:16:47 +02:00
Eitan Turok
0a064dc0fc
Follow up: Fix link in dbrx.md (#30514) 
* Fix link in dbrx.md

* remove "though this may not be up to date"

---------

Co-authored-by: Lysandre Debut <hi@lysand.re>
 2024-05-27 14:57:43 +02:00
Yih-Dar
d7942d9d27
unpin uv (#31055) 
[push-ci-image]

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-05-27 13:47:47 +02:00
Aymeric Roucher
84c4b72ee9
Redirect transformers_agents doc to agents (#31054) 2024-05-27 10:34:14 +02:00
Pablo Montalvo
bdb9106f24
Paligemma- fix devices and dtype assignments (#31008) 
* fix devices and dtype assignments

* [run-slow]paligemma
 2024-05-24 19:02:55 +02:00
Ita Zaporozhets
deba7655e6
Add split special tokens (#30772) 
* seems like `split_special_tokens` is used here

* split special token

* add new line at end of file

* moving split special token test to common tests

* added assertions

* test

* fixup

* add co-author

* passing rest of args to gptsan_japanese, fixing tests

* removing direct comparison of fast and slow models

* adding test support for UDOP and LayoutXLM

* ruff fix

* readd check if slow tokenizer

* modify test to handle bos tokens

* removing commented function

* trigger build

* applying review feedback - updated docstrings, var names, and simplified tests

* ruff fixes

* Update tests/test_tokenization_common.py

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* applying feedback, comments

* shutil temp directory fix

---------

Co-authored-by: Arthur Zucker <arthur.zucker@gmail.com>
Co-authored-by: Ita Zaporozhets <itazaporozhets@Itas-MBP.localdomain>
Co-authored-by: itazap <itazap@users.noreply.github.com>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
Co-authored-by: Ita Zaporozhets <itazaporozhets@Itas-MacBook-Pro.local>
 2024-05-24 08:38:58 -07:00
BHUVAN M
e5103a76cc
added interpolation for vitmae model in pytorch as well as tf. (#30732) 
* added interpolation for vitmae model in pytorch as well as tf.

* Update modeling_vit_mae.py

irreugalr import fixed

* small changes and proper formatting

* changes suggested in review.

* modified decoder interpolate_func

* arguments and docstring fix

* Apply suggestions from code review

doc fixes

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

---------

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2024-05-24 16:20:09 +01:00
Yih-Dar
a3cdff417b
save the list of new model failures (#31013) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-05-24 15:20:25 +02:00
Younes Belkada
658b849aeb
Quantization / TST: Fix remaining quantization tests (#31000) 
* Fix remaining quant tests

* Update test_quanto.py
 2024-05-24 14:35:59 +02:00
Lucain
fd3c128040
Fix resume_download future warning (#31007) 
* Fix resume_download future warning

* better like this

* Add regression test
 2024-05-24 14:35:40 +02:00
Yih-Dar
acbfaf69cc
allow multi-gpu (#31011) 
* allow multi-gpu

* allow multi-gpu

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-05-24 14:20:06 +02:00
Marc Sun
ae87f9797b
FIX / TST: Fix expected results on Mistral AWQ test (#30971) 
fix awq mistral test
 2024-05-24 14:06:31 +02:00
Fanli Lin
04c7c176d7
[tests] make test_model_parallelism device-agnostic (#30844) 
* enable on xpu

* fix style

* add comment and mps
 2024-05-24 11:51:51 +01:00
Yixiang Gao
42d8dd8716
Perceiver interpolate position embedding (#30979) 
* add test that currently fails

* test passed

* all perceiver passed

* fixup, style, quality, repo-consistency, all passed

* Apply suggestions from code review: default to False + compute sqrt once only

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* fix a minor bracket

* replace dim with self._num_channels

* add arguments to the rest preprocessors

---------

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2024-05-24 11:13:58 +01:00