riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-31 02:02:21 +06:00
Code Issues Actions 33 Packages Projects Releases Wiki Activity
13,175 Commits 887 Branches 232 Tags 2.1 GiB
0c3fdccf2f
Commit Graph

13175 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Matthijs Hollemans
0c3fdccf2f
[WIP] add EnCodec model (#23655) 
* boilerplate stuff

* messing around with the feature extractor

* fix feature extractor

* unit tests for feature extractor

* rename speech to audio

* quick-and-dirty import of Meta's code

* import weights (sort of)

* cleaning up

* more cleaning up

* move encoder/decoder args into config

* cleanup model

* rename EnCodec -> Encodec

* RVQ parameters in config

* add slow test

* add lstm init and test_init

* Add save & load

* finish EncodecModel

* remove decoder_input_values as they are ont used anywhere (not removed from doc yet)

* fix test feature extraction model name

* Add better slow test

* Fix tests

* some fixup and cleaning

* Improve further

* cleaning up quantizer

* fix up conversion script

* test don't pass, _encode_fram does not work

* update tests with output per encode and decode

* more cleanup

* rename _codebook

* remove old config cruft

* ratios & hop_length

* use ModuleList instead of Sequential

* clean up resnet block

* update types

* update tests

* fixup

* quick cleanup

* fix padding

* more styl,ing

* add patrick feedback

* fix copies

* fixup

* fix lstm

* fix shape issues

* fixup

* rename conv layers

* fixup

* fix decoding

* small conv refactoring

* remove norm_params

* simplify conv layers

* rename conv layers

* stuff

* Clean up

* Add padding logic

use padding mask

small conv refactoring

remove norm_params

simplify conv layers

rename conv layers

stuff

add batched test

update

Clean up

merge and update for padding

fix padding

fixup

* clean up more

* clean up more

* More clean ups

* cleanup convolutions

* typo

* fix typos

* fixup

* build PR doc?

* start refactoring docstring

* fix don't pad when no strid and chunk

* update docstring

* update docstring

* nits

* update going to lunch

* update config and model

* fix broken testse (becaue of the config changes)

* fix scale computation

* fixu[

* only return dict if speciefied or if config returns it

* remove todos

* update defaults in config

* update conversion script

* fix doctest

* more docstring + fixup

* nits on batched_tests

* more nits

* Apply suggestions from code review

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* update basxed on review

* fix update

* updaet tests

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* fixup

* add overlap and chunl_length_s

* cleanup feature extraction

* teste edge cases truncation and padding

* correct processor values

* update config encodec, nits

* fix tests

* fixup

* fix 24Hz test

* elle tests are green

* fix fixup

* Apply suggestions from code review

* revert readme changes

* fixup

* add example

* use facebook checkpoints

* fix typo

* no pipeline tests

* use slef.pad everywhere we can

* Apply suggestions from code review

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* update based on review

* update

* update mdx

* fix bug and tests

* fixup

* fix doctest

* remove comment

* more nits

* add more coverage for `test_truncation_and_padding`

* fixup

* add last test

* fix text

* nits

* Update tests/models/encodec/test_modeling_encodec.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* take care of the last comments

* typo

* fix test

* nits

* fixup

* Update src/transformers/models/encodec/feature_extraction_encodec.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

---------

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: arthur.zucker@gmail.com <arthur.zucker@gmail.com>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2023-06-14 18:57:23 +02:00
Sylvain Gugger
26a2ec56d7
Clean up old Accelerate checks (#24279) 
* Clean up old Accelerate checks

* Put back imports
 2023-06-14 12:44:09 -04:00
Wissam Antoun
860d11ff7c
Fix Debertav2 embed_proj (#24205) 
* MLM prediction head output size from embed_size

Take the output size of the dense projection layer from embedding_size instead of hidden_size since there could be a projection of the input embedding into hidden_size if they are different

* project TFDebertaV2 mlm output to embedding size

embedding size can be different that hidden_size, so the final layer needs to project back to embedding size. like in ELECTRA or DeBERTaV3 style pertaining.

This should solve an error that occurs when loading models like "almanach/camemberta-base-generator".

* fix the same issue for reshaping after projection

* fix layernorm size

* add self.embedding_size to scope

* fix embed_proj scope name

* apply the same changes to TF Deberta

* add the changes to deberta

* added self.embedding_size instead of config.embedding_size

* added the same change to debertav2

* added coppied from deberta to deberta2 model

* config.embedding_size fix

* black

* fix deberta config name
 2023-06-14 17:24:53 +01:00
Yih-Dar
a04ebc8b33
Pix2StructImageProcessor requires torch>=1.11.0 (#24270) 
* fix

* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-06-14 17:05:40 +02:00
Sylvain Gugger
8978b696d7
Update check of core deps (#24277) 2023-06-14 10:06:31 -04:00
Patrick von Platen
c4fec38bc7
Adapt Wav2Vec2 conversion for MMS lang identification (#24234) 
* Add conversion for mms lid

* make style
 2023-06-14 16:02:36 +02:00
Joao Gante
4626df5077
TF: CTRL with native embedding layers (#23456) 2023-06-14 14:39:02 +01:00
Yih-Dar
eac8dede83
Skip some TQAPipelineTests tests in past CI (#24267) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-06-14 14:25:24 +02:00
ByronHsu
91b62f5a78
QA doc: import torch before it is used (#24228) 
* import torch before it is used

* style

Signed-off-by: byhsu <byhsu@linkedin.com>

---------

Signed-off-by: byhsu <byhsu@linkedin.com>
Co-authored-by: byhsu <byhsu@linkedin.com>
 2023-06-14 11:23:55 +01:00
TAE YOUNGDON
6ab045d6fe
Fix URL in comment for contrastive loss function (#24271) 
* Update language_modeling.py

in "class TextDatasetForNextSentencePrediction(Dataset)", double considering "self.tokenizer.num_special_tokens_to_add(pair=True)" 

so, i remove self.block_size, and add parameter for "def create_examples_from_document". like "class LineByLineWithSOPTextDataset" do

* Update language_modeling.py

* Fix URL in comment for contrastive loss function
 2023-06-14 11:08:31 +01:00
Sourab Mangrulkar
b89fcccd44
update FSDP save and load logic (#24249) 
* update fsdp save and load logic

* fix

* see if this resolves the failing tests
 2023-06-14 00:49:15 +05:30
Sourab Mangrulkar
e0603d894d
docs wrt using accelerate launcher with trainer (#24250) 
* update docs

* missing part

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* address comments

* address Zach's comment

---------

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2023-06-14 00:31:06 +05:30
Yih-Dar
233113149b
Skip GPT-J fx tests for torch < 1.12 (#24256) 
* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-06-13 20:33:26 +02:00
Matt
3bd1fe4315
Stop storing references to bound methods via tf.function (#24146) 
* Stop storing references to bound methods in tf.functions

* Remove the gc.collect calls now that we resolved the underlying problem

* Remove the default signature from model.serving entirely, big cleanup

* Remove _prune_signature as self.input_signature can prune itself

* Restore serving docstring

* Update int support test to check the input signature

* Make sure other tests also use model.input_signature and not serving.input_signature

* Restore _prune_signature

* Remove the doctest GC now it's no longer needed

* Correct core tests to use the pruned sig

* order lines correctly in core tests

* Add eager_serving back with a deprecation warning
 2023-06-13 19:04:22 +01:00
Matt
b979a2064d
Fix how we detect the TF package (#24255) 
* Fix how we detect the TF package

* Add a comment as a talisman warding against future harm

* Actually put the comment in the right place
 2023-06-13 18:57:50 +01:00
Ivan Reznikov
e64d99fa6b
Update urls in warnings for rich rendering (#24136) 
* fixing typo in url in warnings

* fixing typo in url in warnings

* multi-line fix

* multi-line fix

* Update src/transformers/generation/utils.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* Update src/transformers/generation/flax_utils.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* Update src/transformers/generation/tf_utils.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

---------

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2023-06-13 18:23:30 +01:00
Yih-Dar
cf561d7cf1
Add torch >=1.12 requirement for Tapas (#24251) 
* fix

* fix

* fix

* Update src/transformers/models/tapas/modeling_tapas.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2023-06-13 19:19:40 +02:00
Joao Gante
b1ea6b4bf5
Generate: GenerationConfig can overwrite attributes at from_pretrained time (#24238) 
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2023-06-13 17:59:21 +01:00
Joao Gante
7bb6933b9d
TF: standardize test_model_common_attributes for language models (#23457) 2023-06-13 17:51:37 +01:00
Kashif Rasul
4ed075280c
[Time Series] use mean scaler when scaling is a boolean True (#24237) 
* use mean scaler when scaling is boolean True

* remove debug
 2023-06-13 18:46:05 +02:00
Sylvain Gugger
695928e1e5
Tied params cleanup (#24211) 
* First test

* Add info for all models

* style

* Repo consistency

* Fix last model and cleanup prints

* Repo consistency

* Use consistent function for detecting tied weights
 2023-06-13 11:38:39 -04:00
Sourab Mangrulkar
3723329d01
deprecate use_mps_device (#24239) 2023-06-13 19:48:36 +05:30
Sebastian
3e142cb0f5
fix overflow when training mDeberta in fp16 (#24116) 
* Porting changes from https://github.com/microsoft/DeBERTa/ that hopefully allows for fp16 training of mdeberta

* Updates to deberta modeling from microsoft repo

* Performing some cleanup

* Undoing changes that weren't necessary

* Undoing float calls

* Minimally change the p2c block

* Fix error

* Minimally changing the c2p block

* Switch to torch sqrt

* Remove math

* Adding back the to calls to scale

* Undoing attention_scores change

* Removing commented out code

* Updating modeling_sew_d.py to satisfy utils/check_copies.py

* Missed changed

* Further reduce changes needed to get fp16 working

* Reverting changes to modeling_sew_d.py

* Make same change in TF
 2023-06-13 15:04:27 +01:00
amyeroberts
f91810da88
Safely import pytest in testing_utils.py (#24241) 2023-06-13 14:28:08 +01:00
Nicolas Patry
fdd78d9153
Improving error message when using use_safetensors=True. (#24232) 2023-06-13 15:07:00 +02:00
Yih-Dar
74b846cacf
Update (TF)SamModelIntegrationTest (#24199) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-06-13 14:28:14 +02:00
yuanwu2017
d7389cd201
fix: TextIteratorStreamer cannot work with pipeline (#23641) 
* fix: TextIteratorStreamer cannot work with pipeline

Deepcopying the TextIteratorStreamer object causes the exception.

Signed-off-by: yuanwu <yuan.wu@intel.com>

* Update src/transformers/pipelines/text_generation.py

Got it. I will update the patch.

Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>

* Update src/transformers/pipelines/text_generation.py

Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>

* Update text_generation.py

---------

Signed-off-by: yuanwu <yuan.wu@intel.com>
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
 2023-06-13 10:42:41 +01:00
Sylvain Gugger
70c7994095
Fix README copies 2023-06-12 16:24:27 -04:00
Yih-Dar
41a8fa4e14
Add the number of model test failures to slack CI report (#24207) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-06-12 21:27:10 +02:00
Zach Mueller
4da84008dc
Finish dataloader integration (#24201) 2023-06-12 13:26:17 -04:00
Yih-Dar
0675600a60
Update WhisperForAudioClassification doc example (#24188) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-06-12 19:10:31 +02:00
fxmarty
e5dd7432e7
Remove unnecessary aten::to overhead in llama (#24203) 
* fix dtype init

* fix copies

* fix fixcopies mess

* edit forward as well

* copy
 2023-06-12 12:18:04 -04:00
Yih-Dar
4fe9716a79
Skip RWKV test in past CI (#24204) 
* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-06-12 18:14:15 +02:00
Ethan
f7d80cb3d2
Fix steps bugs in no trainer examples (#24197) 
Fix step bugs in no trainer + load checkpoint + grad acc
 2023-06-12 11:49:55 -04:00
Marc Sun
08ae37c820
Fix _load_pretrained_model (#24200) 
Fix test
 2023-06-12 11:31:06 -04:00
Zach Mueller
ebd94b0f6f
🚨🚨🚨 Replace DataLoader logic for Accelerate in Trainer, remove unneeded tests 🚨🚨🚨 (#24028) 
* Working integration

* Fix failing test

* Revert label host logic

* Bring it back!
 2023-06-12 11:23:37 -04:00
Kihoon Son
dc42a9d76f
🌐 [i18n-KO] Translated tasks_summary.mdx to Korean (#23977) 
* 🌐 [i18n-KO] Translated tasks_summary.mdx to Korean

Co-Authored-By: Hyeonseo Yun <0525yhs@gmail.com>
Co-Authored-By: Wonhyeong Seo <wonhseo@kakao.com>
Co-Authored-By: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-Authored-By: Gabriel Yang <gabrielwithhappy@gmail.com>
Co-Authored-By: Jungnerd <46880056+jungnerd@users.noreply.github.com>
Co-Authored-By: Nayeon Han <nayeon2.han@gmail.com>

* Apply suggestions from code review

Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-authored-by: Hyeonseo Yun <0525yhs@gmail.com>

* Update _toctree.yml

* Delete generation_strategies.mdx

* Delete tasks_explained.mdx

---------

Co-authored-by: Hyeonseo Yun <0525yhs@gmail.com>
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-authored-by: Gabriel Yang <gabrielwithhappy@gmail.com>
Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>
Co-authored-by: Nayeon Han <nayeon2.han@gmail.com>
 2023-06-12 11:07:15 -04:00
Joao Gante
60b69f7de2
Generate: detect special architectures when loaded from PEFT (#24198) 2023-06-12 16:06:20 +01:00
Jacob
97527898da
typo: fix typos in CONTRIBUTING.md and deepspeed.mdx (#24184) 
* typo: fix typos in CONTRIBUTING.md and deepspeed.mdx

* Update CONTRIBUTING.md

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

---------

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2023-06-12 15:43:58 +01:00
Yih-Dar
dadc9fb427
Update GPTNeoXLanguageGenerationTest (#24193) 
* fix

* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-06-12 15:37:12 +02:00
Yih-Dar
a9cdb059a8
Fix device issue in OpenLlamaModelTest::test_model_parallelism (#24195) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-06-12 15:21:27 +02:00
Joao Gante
9f81f4f6dd
Generate: force caching on the main model, in assisted generation (#24177) 2023-06-12 14:10:49 +01:00
Kihoon Son
535f92aea3
[i18n]Translated "attention.mdx" to korean (#23878) 
* [i18n]Translated "attention.mdx" to korean

Co-Authored-By: Hyeonseo Yun <0525yhs@gmail.com>
Co-Authored-By: Wonhyeong Seo <wonhseo@kakao.com>
Co-Authored-By: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-Authored-By: Gabriel Yang <gabrielwithhappy@gmail.com>
Co-Authored-By: Nayeon Han <nayeon2.han@gmail.com>
Co-Authored-By: Jungnerd <46880056+jungnerd@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-authored-by: Hyeonseo Yun <0525yhs@gmail.com>

* Update _toctree.yml

---------

Co-authored-by: Hyeonseo Yun <0525yhs@gmail.com>
Co-authored-by: Wonhyeong Seo <wonhseo@kakao.com>
Co-authored-by: Sohyun Sim <96299403+sim-so@users.noreply.github.com>
Co-authored-by: Gabriel Yang <gabrielwithhappy@gmail.com>
Co-authored-by: Nayeon Han <nayeon2.han@gmail.com>
Co-authored-by: Jungnerd <46880056+jungnerd@users.noreply.github.com>
 2023-06-12 08:59:18 -04:00
AinL
ba64ec07bb
Change ProgressCallback to use dynamic_ncols=True (#24101) 
* Change ProgressCallback to use dynamic_ncols=True

* style: make style

* Revert "style: make style"

This reverts commit dee484904c.

* run make style only trainer_callback
 2023-06-12 08:56:48 -04:00
NielsRogge
93f73a3848
Fix push to hub (#24187) 
Add fix
 2023-06-12 08:51:09 -04:00
Yih-Dar
e26c6f03be
Fix Wav2Vec2 CI OOM (#24190) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-06-12 11:39:04 +02:00
Yih-Dar
8f093fb799
Avoid OOM in doctest CI (#24139) 
* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-06-10 09:47:38 +02:00
Stas Bekman
0d217f428f
[tests] fix bitsandbytes import issue (#24151) 
fix bitsandbytes import issue
 2023-06-09 21:53:11 -07:00
Lysandre Debut
deff5979fe
Tool types (#24032) 
* Tool types

* Tests + fixes

* Isolate types

* Oops

* Review comments + docs

* Tests + docs

* soundfile -> vision
 2023-06-09 13:34:07 -04:00
Freddie Vargus
061580c82c
Fix typo in streamers.py (#24144) 2023-06-09 17:27:46 +01:00