transformers/docs/source/en/tasks
Sanchit Gandhi 1c1c90756d
Add Musicgen (#24109)
* Add Audiocraft

* add cross attention

* style

* add for lm

* convert and verify

* introduce t5

* split configs

* load t5 + lm

* clean conversion

* copy from t5

* style

* start pattern provider

* make generation work

* style

* fix pos embs

* propagate shape changes

* propagate shape changes

* style

* delay pattern: pad tokens at end

* audiocraft -> musicgen

* fix inits

* add mdx

* style

* fix pad token in processor

* override generate and add todos

* add init to test

* undo pattern delay mask after gen

* remove cfg logits processor

* remove cfg logits processor

* remove logits processor in favour of mask

* clean pos embs

* make fix copies

* update readmes

* clean pos emb

* refactor encoder/decoder

* make fix copies

* update conversion

* fix config imports

* update config docs

* make style

* send pattern mask to device

* pattern mask with delay

* recover prompted audio tokens

* fix docstrings

* laydown test file

* pattern edge case

* remove t5 ref

* add processing class

* config refactor

* better pattern comment

* check if mask is not present

* check if mask is not present

* refactor to auto class

* remove encoder configs

* fix processor

* processor import

* start updating conversion

* start updating tests

* make style

* convert t5, encodec, lm

* convert as composite

* also convert processor

* run generate

* classifier free gen

* comments and clean up

* make style

* docs for logit proc

* docstring for uncond gen

* start lm tests

* work tests

* let the lm generate

* refactor: reshape inside forward

* undo greedy loop changes

* from_enc_dec -> from_sub_model

* fix input id shapes in docstrings

* Apply suggestions from code review

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* undo generate changes

* from sub model config

* Update src/transformers/models/musicgen/modeling_musicgen.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* make generate work again

* generate uncond -> get uncond inputs

* remove prefix allowed tokens fn

* better error message

* logit proc checks

* Apply suggestions from code review

Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>

* make decoder only tests work

* composite fast tests

* make style

* uncond generation

* feat extr padding

* make audio prompt work

* fix inputs docstrings

* unconditional inputs: dict -> model output

* clean up tests

* more clean up tests

* make style

* t5 encoder -> auto text encoder

* remove comments

* deal with frames

* fix auto text

* slow tests

* nice mdx

* remove can generate

* todo - hub id

* convert m/l

* make fix copies

* only import generation with torch

* ignore decoder from tests

* don't wrap uncond inputs

* make style

* cleaner uncond inputs

* add example to musicgen forward

* fix docs

* ignore MusicGen Model/ForConditionalGeneration in auto mapping

* add doc section to toctree

* add to doc tests

* add processor tests

* fix push to hub in conversion

* tips for decoder only loading

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* fix conversion for s / m / l checkpoints

* import stopping criteria from module

* remove from pipeline tests

* fix uncond docstring

* decode audio method

* fix docs

* org: sanchit-gandhi -> facebook

* fix max pos embeddings

* remove auto doc (not compatible with shapes)

* bump max pos emb

* make style

* fix doc

* fix config doc

* fix config doc

* ignore musicgen config from docstring

* make style

* fix config

* fix config for doctest

* consistent from_sub_models

* don't automap decoder

* fix mdx save audio file

* fix mdx save audio file

* processor batch decode for audio

* remove keys to ignore

* update doc md

* update generation config

* allow changes for default generation config

* update tests

* make style

* fix docstring for uncond

* fix processor test

* fix processor test

---------

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-06-29 14:48:59 +01:00
..
asr.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
audio_classification.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
document_question_answering.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
image_captioning.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
image_classification.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
language_modeling.md Add Musicgen (#24109) 2023-06-29 14:48:59 +01:00
masked_language_modeling.md Update masked_language_modeling.md (#24560) 2023-06-28 17:54:20 -04:00
monocular_depth_estimation.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
multiple_choice.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
object_detection.md Update old existing feature extractor references (#24552) 2023-06-29 10:17:36 +01:00
question_answering.md [T5] Add T5ForQuestionAnswering and MT5ForQuestionAnswering (#24481) 2023-06-27 10:07:06 -04:00
semantic_segmentation.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
sequence_classification.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
summarization.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
text-to-speech.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
token_classification.md Update token_classification.md (#24484) 2023-06-26 08:42:38 -04:00
translation.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
video_classification.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
zero_shot_image_classification.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00
zero_shot_object_detection.md Migrate doc files to Markdown. (#24376) 2023-06-20 18:07:47 -04:00