riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-13 01:30:04 +06:00
Code Issues Actions 7 Packages Projects Releases Wiki Activity
15,053 Commits 863 Branches 226 Tags 1.9 GiB
3f9f749325
Commit Graph

15053 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Isaac Chung
d2f06dfffc
[docstring] Fix docstring for CLIPImageProcessor (#26676) 
fix docstring for CLIPImageProcessor
 2023-10-09 14:22:44 +02:00
Isaac Chung
3763101f85
[docstring] Fix docstring CLIP configs (#26677) 
* fix docstrings for CLIP configs

* black formatted
 2023-10-09 12:34:01 +02:00
tom white
c7f01beece
fix typos in idefics.md (#26648) 
* fix typos in idefics.md

Two typos found in reviewing this documentation.

1) max_new_tokens=4, is not sufficient to generate "Vegetables" as indicated - you will get only "Veget". (incidentally - some mention of how to select this value might be useful as it seems to change in each example)

2) inputs = processor(prompts, return_tensors="pt").to(device) as inputs need to be on the same device (as they are in all other examples on the page)

* Update idefics.md

Change device to cuda explicitly to match other examples
 2023-10-09 12:18:02 +02:00
Yih-Dar
740fc6a1da
Avoid CI OOM (#26639) 
fix avoid oom

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-10-09 11:42:08 +02:00
D. Carpintero
8835bff6a0
fix links in README.md for the GPT, GPT-2, and Llama2 Models (#26640) 
* fix OpenAI GPT, GPT-2 links

* fix Llama2 link
 2023-10-09 11:34:44 +02:00
Shreyas S
86a4e5a96b
Fixed malapropism error (#26660) 
Update test_integration.py

Fixed malapropism clone>copy
 2023-10-09 11:04:57 +02:00
NielsRogge
2629c8f36a
[DINOv2] Convert more checkpoints (#26177) 
* Convert checkpoints

* Update doc test

* Address comment
 2023-10-09 09:58:04 +02:00
Jabasukuriputo Wang
897a826d83
docs(zh): review and punctuation & space fix (#26627) 2023-10-06 09:24:28 -07:00
Yih-Dar
360ea8fc72
[docstring] Fix docstring for AlbertConfig (#26636) 
example fix docstring

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-10-06 17:36:22 +02:00
Arthur
9ad815e412
[LlamaTokenizerFast] Adds edge cases for the template processor (#26606) 
* make sure eos and bos are properly handled for fast tokenizer

* fix code llama as well

* nits

* fix the conversion script as well

* fix failing test
 2023-10-06 16:40:54 +02:00
statelesshz
27597fea07
remove SharedDDP as it is deprecated (#25702) 
* remove SharedDDP as it was drepracated

* apply review suggestion

* make style

* Oops,forgot to remove the compute_loss context manager in Seq2SeqTrainer.

* remove the unnecessary conditional statement

* keep the logic of IPEX

* clean code

* mix precision setup & make fixup

---------

Co-authored-by: statelesshz <jihuazhong1@huawei.com>
 2023-10-06 16:03:11 +02:00
Yih-Dar
e840aa67e8
Fix failing MusicgenTest .test_pipeline_text_to_audio (#26586) 
* fix

* fix

* Fix

* Fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-10-06 15:53:59 +02:00
rui-ren
87499420bf
fix RoPE t range issue for fp16 (#26602) 2023-10-06 12:04:54 +01:00
Matt
ea52ed9dc8
Update chat template docs with more tips on writing a template (#26625) 2023-10-06 12:04:40 +01:00
fxmarty
64845307b3
Remove unnecessary unsqueeze - squeeze in rotary positional embedding (#26162) 
* remove unnecessary unsqueeze-squeeze in llama

* correct other models

* fix

* revert gpt_neox_japanese

* fix copie

* fix test
 2023-10-06 18:25:15 +09:00
Tianqi Liu
65aabafe2f
Update tokenization_code_llama_fast.py (#26576) 
* Update tokenization_code_llama_fast.py

* Update test_tokenization_code_llama.py

* Update test_tokenization_code_llama.py
 2023-10-06 10:49:02 +02:00
Towdo
af38c837ee
Fixed inconsistency in several fast tokenizers (#26561) 2023-10-06 10:40:47 +02:00
Ramiro Leal-Cavazos
8878eb1bd9
Remove unnecessary views of position_ids (#26059) 
* Remove unnecessary `view` of `position_ids` in `modeling_llama`

When `position_ids` is `None`, its value is generated using
`torch.arange`, which creates a tensor of size `(seq_length +
past_key_values_length) - past_key_values_length = seq_length`. The
tensor is then unsqueezed, resulting in a tensor of shape `(1,
seq_length)`. This means that the last `view` to a tensor of shape
`(-1, seq_length)` is a no-op.

This commit removes the unnecessary view.

* Remove no-op `view` of `position_ids` in rest of transformer models
 2023-10-06 10:28:00 +02:00
Yih-Dar
75a33d60f2
Don't install pytorch-quantization in Doc Builder docker file (#26622) 
Fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-10-05 16:57:50 +02:00
Maria Khalusova
18fbeec824
[docs] Update to scripts building index.md (#26546) 
* build the table in index.md with links to the model_doc

* removed list generation on index.md

* fixed missing models

* make style
 2023-10-05 10:20:41 -04:00
Yih-Dar
9d20601259
Fix transformers-pytorch-gpu docker build (#26615) 
Fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-10-05 15:33:35 +02:00
eajechiloae
9e78c9acfb
Don't close ClearML task if it was created externally (#26614) 
don't close clearml task if it was created externally
 2023-10-05 15:33:05 +02:00
Marvin Gabler
0a3b9d02fe
#26566 swin2 sr allow in out channels (#26568) 
* feat: close #26566, changed model & config files to accept arbitary in and out channels

* updated docstrings

* fix: linter error

* fix: update Copy docstrings

* fix: linter update

* fix: rename num_channels_in to num_channels to prevent breaking changes

* fix: make num_channels_out None per default

* Update src/transformers/models/swin2sr/configuration_swin2sr.py

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* fix: update tests to include num_channels_out

* fix:linter

* fix: remove normalization with precomputed rgb values when #input_channels!=#output_channels

---------

Co-authored-by: marvingabler <marvingabler@outlook.de>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
 2023-10-05 15:20:38 +02:00
Younes Belkada
e6d250e4cd
[core] fix silent bug keep_in_fp32 modules (#26589) 
* fix silent bug `keep_in_fp32` modules

* final fix

* added a common test.

* Trigger CI

* revert
 2023-10-05 14:44:31 +02:00
Charles Bensimon
19f0b7dd02
Make ModelOutput serializable (#26493) 
* Make `ModelOutput` serializable

Original PR from diffusers : https://github.com/huggingface/diffusers/pull/5234

* Black
 2023-10-05 11:08:44 +02:00
Yih-Dar
54e17a15dc
Fix failing tests on main due to torch 2.1 (#26607) 
* fix

* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-10-05 10:27:05 +02:00
Yun Dai
2ab76c2c4f
[Falcon] Set use_cache=False before creating presents which relies on use_cache (#26328) 
* Set `presents=None` when `use_cache` is set to False for activation ckpt

* Update modeling_falcon.py

* fix black
 2023-10-05 10:18:27 +02:00
Arthur
253f9a3f97
[GPTNeoX] Faster rotary embedding for GPTNeoX (based on llama changes) (#25830) 
* Faster rotary embedding for GPTNeoX

* there might be un-necessary moves from device

* fixup

* fix dtype issue

* add copied from statements

* fox copies

* oupsy

* add copied from Llama for scaled ones as well

* fixup

* fix

* fix copies
 2023-10-05 10:05:39 +02:00
Arthur
b4e66d7a67
[ NougatProcessor] Fix the default channel (#26608) 
fix
 2023-10-05 09:38:08 +02:00
Yeyang
43bfd093e1
add zh translation for installation (#26084) 
* translate installation to zh

* fix translation typo
 2023-10-04 09:39:02 -07:00
Sanchit Gandhi
2d8ee9817c
[Wav2Vec2] Fix tokenizer set lang (#26349) 
* fix wav2vec2 doctest

* suggestion

* fix

* final fix

* revert since we need AddedTokens
 2023-10-04 17:12:09 +01:00
Galland
f9ab07f920
Update mistral.md to update 404 link (#26590) 2023-10-04 17:48:11 +02:00
Arthur
c037b2e340
skip flaky hub tests (#26594) 
skip flaky
 2023-10-04 17:47:55 +02:00
Soyoung Yoon
ca7912d191
Fix encoder->decoder typo bug in convert_t5x_checkpoint_to_pytorch.py (#26587) 
Fix bug in convert_t5x_checkpoint_to_pytorch.py
 2023-10-04 17:34:32 +02:00
Matt
8b03615b7b
Fix embarrassing typo in the doc chat template! (#26596) 2023-10-04 16:28:53 +01:00
dg845
9deb18ca1a
Add # Copied from statements to audio feature extractors that use the floats_list function (#26581) 
Add # Copied from statements to audio feature extractors that use the floats_list function.
 2023-10-04 17:09:48 +02:00
Sanchit Gandhi
0a49f909bc
[Mistral] Update config docstring (#26593) 
* fix copies

* fix missing docstring

* make style

* oops
 2023-10-04 16:02:34 +01:00
Phuc Van Phan
6015f91a5a
refactor: change default block_size (#26229) 
* refactor: change default block_size

* fix: return tf to origin

* fix: change files to origin

* rebase

* rebase

* rebase

* rebase

* rebase

* rebase

* rebase

* rebase

* refactor: add min block_size to files

* reformat: add min block_size for run_clm tf
 2023-10-04 15:31:38 +01:00
Matt
8b46c5bcfc
Add add_generation_prompt argument to apply_chat_template (#26573) 
* Add add_generation_prompt argument to apply_chat_template

* Add add_generation_prompt argument to apply_chat_template and update default templates

* Fix typo

* Add generation prompts section to chat templating guide

* Add generation prompts section to chat templating guide

* Minor style fix
 2023-10-04 15:15:29 +01:00
Sylvain Gugger
03af4c42a6
Docstring check (#26052) 
* Fix number of minimal calls to the Hub with peft integration

* Alternate design

* And this way?

* Revert

* Nits to fix

* Add util

* Print when changes are made

* Add list to ignore

* Add more rules

* Manual fixes

* deal with kwargs

* deal with enum defaults

* avoid many digits for floats

* Manual fixes

* Fix regex

* Fix regex

* Auto fix

* Style

* Apply script

* Add ignored list

* Add check that templates are filled

* Adding to CI checks

* Add back semi-fix

* Ignore more objects

* More auto-fixes

* Ignore missing objects

* Remove temp semi-fix

* Fixes

* Update src/transformers/models/pvt/configuration_pvt.py

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* Update utils/check_docstrings.py

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* Update src/transformers/utils/quantization_config.py

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* Deal with float defaults

* Fix small defaults

* Address review comment

* Treat

* Post-rebase cleanup

* Address review comment

* Update src/transformers/models/deprecated/mctct/configuration_mctct.py

Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>

* Address review comment

---------

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
 2023-10-04 15:13:37 +02:00
Bharat Ramanathan
122b2657f8
feat: add trainer label to wandb run upon initialization (#26466) 2023-10-04 14:57:41 +02:00
statelesshz
4fdf47cd3c
Extend Trainer to enable Ascend NPU to use the fused Adamw optimizer when training (#26194) 2023-10-04 14:57:11 +02:00
dependabot[bot]
fc296f419e
Bump pillow from 9.3.0 to 10.0.1 in /examples/research_projects/decision_transformer (#26580) 
Bump pillow in /examples/research_projects/decision_transformer

Bumps [pillow](https://github.com/python-pillow/Pillow) from 9.3.0 to 10.0.1.
- [Release notes](https://github.com/python-pillow/Pillow/releases)
- [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst)
- [Commits](https://github.com/python-pillow/Pillow/compare/9.3.0...10.0.1)

---
updated-dependencies:
- dependency-name: pillow
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-10-04 11:52:46 +02:00
김준재_T3056
2f3ea08a07
docs: feat: add clip notebook resources from OSSCA community (#26505) 2023-10-03 11:20:22 -07:00
Lysandre Debut
5c66378cea
[Tokenizers] Skip tests temporarily (#26574) 
* Skip tests temporarily

* style

* Add additional test
 2023-10-03 19:43:42 +02:00
Jungnerd
2c7b26f508
🌐 [i18n-KO] Translated semantic_segmentation.md to Korean (#26515) 
* docs: ko: sementic_segmentation.md

* feat: manual draft

* fix: manual edits

* fix: resolve suggestions

Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>

* fix: resolve suggestions

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* fix: edit the title

---------

Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2023-10-03 10:25:50 -07:00
Sanchit Gandhi
57f44dc428
[Whisper] Allow basic text normalization (#26149) 
* [Whisper] Allow basic text normalization

* up

* style copies
 2023-10-03 17:57:16 +01:00
Lysandre
bd6205919a v4.35.0.dev0 2023-10-03 16:54:37 +02:00
Arthur
c26b2a29e5
[Nougat] from transformers import * (#26562) 
* remove unprotected import to PIL

* cleanup

---------

Co-authored-by: Lysandre <lysandre@huggingface.co>
 2023-10-03 16:32:12 +02:00
Younes Belkada
2aef9a9601
[PEFT] Final fixes (#26559) 
* fix issues with PEFT

* logger warning futurewarning issues

* fixup

* adapt from suggestions

* oops

* rm test
 2023-10-03 14:53:09 +02:00