riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-31 02:02:21 +06:00
Code Issues Actions 33 Packages Projects Releases Wiki Activity
13,233 Commits 887 Branches 232 Tags 2.1 GiB
eb849f6604
Commit Graph

13233 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Sylvain Gugger
eb849f6604
Migrate doc files to Markdown. (#24376) 
* Rename index.mdx to index.md

* With saved modifs

* Address review comment

* Treat all files

* .mdx -> .md

* Remove special char

* Update utils/tests_fetcher.py

Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>

---------

Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
 2023-06-20 18:07:47 -04:00
Patrick von Platen
b0513b013b
[Wav2Vec2 - MMS] Correct directly loading adapters weights (#24335) 
* Correct direct lang loading

* correct more

* revert black

* Use tie weights instead=

* add tests

* add tests

* make style
 2023-06-20 19:39:52 +02:00
Arthur
e5c760d636
[GPTNeoX] Nit in config (#24349) 
* add raise value error for attention size

* nits to fix test_config

* style
 2023-06-20 19:19:19 +02:00
Arthur
c2882403c4
[Whisper Docs] Nits (#24367) 
* nits

* config doc did not match

* Apply suggestions from code review

Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>

---------

Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
 2023-06-20 19:18:52 +02:00
Yih-Dar
83dc5762e7
Skip a tapas (tokenization) test in past CI (#24378) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-06-20 18:35:45 +02:00
Yih-Dar
297d769d0e
Better test name and enable pipeline test for pix2struct (#24377) 
* best test name forever

* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-06-20 18:29:30 +02:00
Aaron Pham
6950f70b38
style: add BitsAndBytesConfig __repr__ function (#24331) 
* style: add repr to BitsAndBytesConfig

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: update pattern for __repr__

implement diff dict for __repr__ of BitsAndBytesConfig

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-06-20 12:26:08 -04:00
Arthur
7feba74400
[Tokenizer doc] Clarification about add_prefix_space (#24368) 
* nits

* more details

* fixup

* Apply suggestions from code review

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

---------

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2023-06-20 18:22:00 +02:00
Yih-Dar
0527c1c0ea
Add a check in ImageToTextPipeline._forward (#24373) 
* fix

* fix

* fix

* Update src/transformers/pipelines/image_to_text.py

Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com>

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com>
 2023-06-20 18:07:34 +02:00
Sylvain Gugger
dc4449918d
Rename test to be more accurate (#24374) 2023-06-20 11:54:55 -04:00
Sylvain Gugger
a6b4d1ad83
Remove print statement 2023-06-20 11:14:29 -04:00
Sanchit Gandhi
6c1344444a
[Whisper] Make tests faster (#24105) 2023-06-20 16:01:56 +01:00
Sanchit Gandhi
f924df3c7e
[modelcard] add audio classification to task list (#24363) 2023-06-20 14:01:17 +01:00
Yih-Dar
c23d131eab
Update tiny models for pipeline testing. (#24364) 
* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-06-20 14:43:10 +02:00
Matt
56efbf4301
TensorFlow CI fixes (#24360) 
* Fix saved_model_creation_extended

* Skip the BLIP model creation test for now

* Fix TF SAM test

* Fix longformer tests

* Fix Wav2Vec2

* Add a skip for XLNet

* make fixup

* make fix-copies

* Add comments
 2023-06-20 12:59:21 +01:00
Llohann Dallagnol Speranca
183f442ba8
Fix resuming PeftModel checkpoints in Trainer (#24274) 
* Fix resuming checkpoints for PeftModels

Fix an error occurred when resuming a PeftModel from a training checkpoint. That was caused since PeftModel.pre_trained saves only adapter-related data while _load_from_checkpoint was expecting a torch sved model. This PR fix this issue and allows the adapter checkpoint to be loaded.

Resolves: #24252

* fix last comment

* fix nits

---------

Co-authored-by: younesbelkada <younesbelkada@gmail.com>
 2023-06-20 13:57:08 +02:00
Matt
0875b2509a
Allow passing kwargs through to TFBertTokenizer (#24324) 2023-06-20 12:49:06 +01:00
Denis Ismailaj
cfc838dd4d
Respect explicitly set framework parameter in pipeline (#24322) 
* Respect framework parameter

* Move check to pipeline()

* Add check inside infer_framework_load_model again
 2023-06-20 11:43:52 +01:00
Quentin Gallouédec
c5454eba9e
Fix the order in GPTNeo's docstring (#24358) 
* Fix arg sort in docstring

* further order fix

* make style
 2023-06-19 18:59:35 +01:00
Ritesh Ghorse
20273ee214
[Doc Fix] Fix model name path in the transformers doc for AutoClasses (#24329) 
fix model name path

Co-authored-by: Ritesh Ghorse <riteshghorse@Riteshs-Air.attlocal.net>
 2023-06-19 17:26:55 +01:00
Aaron Pham
c003c8cb52
docs: add BentoML to awesome-transformers (#24344) 
* docs: add BentoML to awesome-transformers

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

* chore: add the project to the bottom of the line

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

---------

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
 2023-06-19 12:17:30 -04:00
Gema Parreño
52c4276e44
Fix link to documentation in Install from Source (#24336) 
Update __init__.py

Fix link to documentation to install Transformers from source 
Probably the title changed at some point from 'Installing' to 'Install'
 2023-06-19 17:12:55 +01:00
amyeroberts
7e71eb2ef7
Fix ImageGPT doctest (#24353) 
Fix doctest
 2023-06-19 15:23:29 +01:00
Yih-Dar
a4de24f691
Make AutoFormer work with previous torch version (#24357) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-06-19 16:02:06 +02:00
Vineel Pratap
7761b1893a
Update MMS integration docs (#24311) 
* Update mms.mdx

* Update mms.mdx

* Update docs/source/en/model_doc/mms.mdx

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Update mms.mdx

* Update docs/source/en/model_doc/mms.mdx

Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>

---------

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
 2023-06-19 14:49:01 +01:00
Yih-Dar
5fca839fef
Fix device issue in SwitchTransformers (#24352) 
* fix

* Update src/transformers/models/switch_transformers/modeling_switch_transformers.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2023-06-19 15:06:05 +02:00
Matěj Kripner
3b5a56e595
Fix KerasMetricCallback: pass generate_kwargs even if use_xla_generation is False (#24333) 
* Fix `KerasMetricCallback`: always pass `generate_kwargs`.

* Reformat code using Black.
 2023-06-19 12:51:25 +01:00
Yih-Dar
0b259a3b7e
Clean up disk sapce during docker image build for transformers-pytorch-gpu (#24346) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-06-19 12:54:02 +02:00
Yih-Dar
691b60db90
byebye Hub connection timeout (#24350) 
byebye timeout

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-06-19 12:50:20 +02:00
Yih-Dar
17e3e7d686
pin apex to a speicifc commit (for DeepSpeed CI docker image) (#24351) 
* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-06-19 12:48:53 +02:00
Sohyun Sim
3c124df579
🌐 [i18n-KO] Fixed tutorial/preprocessing.mdx (#24156) 
* fix: revise translations

* fix: resolve suggestions

Co-authored-by: Hyeonseo Yun <0525yhs@gmail.com>

---------

Co-authored-by: Hyeonseo Yun <0525yhs@gmail.com>
 2023-06-19 11:43:57 +01:00
Xiaoyang Sun
881c0df952
error bug on saving distributed optim state when using data parallel (#24108) 
Update checkpoint_reshaping_and_interoperability.py
 2023-06-19 16:04:21 +05:30
Teven
ee88ae5994
Adding ddp_broadcast_buffers argument to Trainer (#24326) 
adding ddp_broadcast_buffers argument
 2023-06-16 15:14:03 -04:00
Matt
9138995025
Add test for proper TF input signatures (#24320) 
* Add test for proper input signatures

* No more signature pruning

* Test the dummy inputs are valid too

* fine-tine -> fine-tune

* Fix indent in test_dataset_conversion
 2023-06-16 17:03:13 +01:00
amyeroberts
bdfd57d1d1
Fix ImageGPT doc example (#24317) 
* Fix ImageGPT doc example

* Update src/transformers/models/imagegpt/image_processing_imagegpt.py

* Fix types
 2023-06-16 17:01:22 +01:00
Sylvain Gugger
096f2cf126
Tied weights load (#24310) 
* Use tied weight keys

* More

* Fix tied weight missing warning

* Only give info on unexpected keys with different classes

* Deal with empty archs

* Fix tests

* Refine test
 2023-06-16 10:55:42 -04:00
Nicolas Patry
61ffdeba38
Fix ner average grouping with no groups (#24319) 
Fixes #https://github.com/huggingface/transformers/issues/24314
 2023-06-16 16:43:19 +02:00
Matt
3403712958
Big TF test cleanup (#24282) 
* Fix one BLIP arg not being optional, remove misspelled arg

* Remove the lxmert test overrides and just use the base test_saved_model_creation

* saved_model_creation fixes and re-enabling tests across the board

* Remove unnecessary skip

* Stop caching sinusoidal embeddings in speech_to_text

* Fix transfo_xl compilation

* Fix transfo_xl compilation

* Fix the conditionals in xglm

* Set the save spec only when building

* Clarify comment

* Move comment correctly

* Correct embeddings generation for speech2text

* Mark RAG generation tests as @slow

* Remove redundant else:

* Add comment to clarify the save_spec line in build()

* Fix size tests for XGLM at last!

* make fixup

* Remove one band_part operation

* Mark test_keras_fit as @slow
 2023-06-16 15:40:49 +01:00
Yih-Dar
896a58de15
Byebye pytorch 1.9 (#24080) 
byebye

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-06-16 16:38:23 +02:00
Matt
62d71f4083
Fix functional TF Whisper and modernize tests (#24301) 
* Revert whisper change and modify the test_compile_tf_model test

* make fixup

* Tweak test slightly

* Add functional model saving to test

* Ensure TF can infer shapes for data2vec

* Add override for efficientformer

* Mark test as slow
 2023-06-16 14:43:43 +01:00
Arthur
ba3fb4b8d7
[SwitchTransformers] Fix return values (#24300) 
* clean history

* remove other changes

* fix

* fix coipes
 2023-06-16 15:40:33 +02:00
Sayed Qaiser Ali
0b7b4429c7
Update test versions on README.md (#24307) 
Update README.md

Updated the tested versions
 2023-06-15 18:01:11 +01:00
Yih-Dar
6134b9b4c7
Make can_generate as class method (#24299) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-06-15 18:31:38 +02:00
jprivera44
e45bc14350
Beam search type (#24288) 
* test check in

* adding in type hint fix on beam search

* fixed code quality issue
 2023-06-15 16:48:02 +01:00
Belladore
1a113fcf65
Update tokenizer_summary.mdx (grammar) (#24286) 2023-06-15 16:31:47 +01:00
hitchhicker
c3ca346b49
[Docs] Fix the paper URL for MMS model (#24302) 
Fix the paper URL for MMS model
 2023-06-15 15:45:49 +01:00
Sanchit Gandhi
4124a09f8b
[EnCodec] Changes for 32kHz ckpt (#24296) 
* [EnCodec] Changes for 32kHz ckpt

* Update src/transformers/models/encodec/convert_encodec_checkpoint_to_pytorch.py

* Update src/transformers/models/encodec/convert_encodec_checkpoint_to_pytorch.py
 2023-06-15 14:36:19 +01:00
Sourab Mangrulkar
01b55779d3
deepspeed init during eval fix (#24298) 
* deepspeed init during eval fix

* commit suggestions

Co-Authored-By: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

---------

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2023-06-15 18:47:09 +05:30
Cooper
6a081c512a
Update README_zh-hans.md (#24181) 
* Update README_zh-hans.md

update document link

* Update README_zh-hans.md

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

---------

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2023-06-15 13:50:40 +01:00
Patrick von Platen
604a21b1e6
[Docs] Improve docs for MMS loading of other languages (#24292) 
* Improve docs

* Apply suggestions from code review

* upload readme

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

---------

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2023-06-15 14:29:32 +02:00