transformers/docs/source/en
Arthur c236a62172
[CLAP] Add CLAP to the library (#21370)
* add model like clip

* update

* text model ok

* clap text works

* some refactor

- `CLAPVision` to `CLAPAudio`
- refactor kwargs of audio modules

* more refactor

* more refactor

* more refactor

* correct fusion

* more refactor

* new modules

* add basic processor

* fixup

* remove whisper copioed from

* audio logits match

* add doc

* correct filters mel and add maxlength

* style

* few fixes

* forward passes

* fixup

* fixup

* some clean up

* remove mels form the dictionnary

* pad after the repeat

* update padding when dsmaller

* fix padding

* style

* use swin patch merging

* use copied from swin

* processor with any tokenizer

* more copied from

* some clean up

* more refactor

* fix mel when rand_trunc

* style

* remove unused imports

* update processing

* remove image processing tests

* add testing fiel

* fixmodeling issues

* replace with `is_longer`

* clap in serialization

* more refactor

* `make fixup`

* make fixup

* fix feature extractor

* update test feature extractor

* `make fixup`

* clean up config

* more clean up

* more cleanup

* update tests

* refactor tests and inits

* removeCLAP vision config

* remove CLAP from image procssing auto and dummy vision objects

* update inits

* style

* re order classes in modeling clap

* Use roberta tokenizer as the other weights are not open sourced

* small cleaup

* remove tokenization CLAP

* processor tokenizr is roberta

* update feature extraction doc

* remove vclap from model zero shot

* update f_min and f_max to frequency_xx

* some changes

- fix modeling keys
- add `is_longer` in the forward pass
- make fixup

* make fixup

* consistent behavior ebtween rand_crop and fusion

* add numpy resize and bilinear and documentation

* move resizing to image utils

* clean feature extraction

* import resize from correct file

* resize in image transforms

* update

* style

* style

* nit

* remove unused arguments form the feature extractor

* style

* few fixes + make fixup

* oops

* fix more tests

* add zero shot audio classification pipeline

* update zeroshot classification pipeline

* fixup

* fix copies

* all CI tests pass

* make fixup + fix docs

* fix docs

* fix docs

* update tests pip;eline

* update zero shot pipeline

* update feature extraction clap

* update tokenization auto

* use nested simplify

* update pipeline tests

* Apply suggestions from code review

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* split in two lines

* fixes

* refactor

* clean up

* add integration tests

* update config docstring

* style

* update processor

* fix processor test

* fix feat extractor tests

* update docs

* Apply suggestions from code review

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* fix readmes

* fix tips

* Update src/transformers/models/auto/configuration_auto.py

* update doc and remove todo -> properly explained

* fix idx and typo

* typoe

* cleanup config

* cleanup tests, styles and doc

* ignore docstyle on image transform

* add conversion script

* remove the `clap` indx in favor of `CLAP`

* update __init

* nits

* Update src/transformers/pipelines/__init__.py

* fix bug

* clarifiy config

* fix copy

* fix init

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* fix model output

* fix comment

* make fixup

* make fixup

* rename to `Clap`

* replace to `Clap`

* replace to `Clap`

* repo consistency

* again repo-consistency

* make fixup

* Apply suggestions from code review

Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>

* add config

* changes

* update conversion

* Apply suggestions from code review

Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>

* remove unused function

* update based on code reviews

* style

* more comments

* cleanup

* clean up

* style

* apply suggestions

* Empty commit

* pipeline will be added in a different PR

* update calls to audio utils functions

* update pipeline init

* style

* style

* styling again

* use pad

* fix repo-consistency

* update utils and add doc for audio utils

* clean up resize by using torch. update inits accordingly

* style

* CLap's  tokenizer is RobertA

* add audio utils to internal toctreee

* update totctree

* style

* update documentation and normalize naming accross audio utils and feature extraction clap

* style

* clean up

* update doc and typos

* fix doctest

* update modelin code, got rid of a lot of reshaping

* style on added doc audio utils

* update modeling clap

* style

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* docstringvariables with CLAP

* rename key

* update modeling CLAP

* update audio utils docstring

* update processing clap

* fix readmes

* fix toctree

* udpate configuration clap

* fix init

* make fixup

* fix

* fix

* update naming

* update

* update checkpoint path

* Apply suggestions from code review

* Major refactoring

* Update src/transformers/models/clap/configuration_clap.py

* merge

---------

Co-authored-by: younesbelkada <younesbelkada@gmail.com>
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
2023-02-16 20:59:27 +01:00
..
internal [CLAP] Add CLAP to the library (#21370) 2023-02-16 20:59:27 +01:00
main_classes [deepspeed] performance docs (#21573) 2023-02-13 10:29:12 -08:00
model_doc [CLAP] Add CLAP to the library (#21370) 2023-02-16 20:59:27 +01:00
tasks Add Ernie-M Model to huggingface (#21349) 2023-02-15 09:24:56 -05:00
_config.py Enable doc in Spanish (#16518) 2022-04-04 10:25:46 -04:00
_toctree.yml [CLAP] Add CLAP to the library (#21370) 2023-02-16 20:59:27 +01:00
accelerate.mdx update to use interlibrary links instead of Markdown (#18500) 2022-08-08 10:53:52 -05:00
add_new_model.mdx 🚨🚨🚨 Enforce single model initialization (#21431) 2023-02-09 15:46:26 -05:00
add_new_pipeline.mdx Spanish translation of asr.mdx and add_new_pipeline.mdx (#20569) 2022-12-12 09:23:23 -05:00
add_tensorflow_model.mdx docs: Resolve many typos in the English docs (#20088) 2022-11-07 09:19:04 -05:00
attention.mdx Refactor model summary (#21408) 2023-02-15 10:35:14 -08:00
autoclass_tutorial.mdx Update doc examples feature extractor -> image processor (#20501) 2022-11-30 14:50:55 +00:00
benchmarks.mdx Enable doc in Spanish (#16518) 2022-04-04 10:25:46 -04:00
bertology.mdx Enable doc in Spanish (#16518) 2022-04-04 10:25:46 -04:00
big_models.mdx docs: Resolve many typos in the English docs (#20088) 2022-11-07 09:19:04 -05:00
community.mdx Enable doc in Spanish (#16518) 2022-04-04 10:25:46 -04:00
contributing.md Enable doc in Spanish (#16518) 2022-04-04 10:25:46 -04:00
converting_tensorflow_models.mdx Docs - Guide to add a new TensorFlow model (#19256) 2022-09-30 20:30:38 +01:00
create_a_model.mdx Documentation code sample fixes (#21302) 2023-01-25 11:33:39 -05:00
custom_models.mdx Replace awkward timm link with the expected one (#20109) 2022-11-07 13:57:39 -05:00
debugging.mdx Spanish translation of the file debugging.mdx (#20566) 2022-12-12 10:38:56 -05:00
fast_tokenizers.mdx Enable doc in Spanish (#16518) 2022-04-04 10:25:46 -04:00
generation_strategies.mdx Add: An introductory guide for text generation (#21090) 2023-01-17 12:23:22 -05:00
glossary.mdx Update doc examples feature extractor -> image processor (#20501) 2022-11-30 14:50:55 +00:00
hpo_train.mdx update doc for perf_train_cpu_many (#19506) 2022-10-11 22:54:19 -04:00
index.mdx [CLAP] Add CLAP to the library (#21370) 2023-02-16 20:59:27 +01:00
installation.mdx Move cache folder to huggingface/hub for consistency with hf_hub (#18492) 2022-08-05 13:14:00 -04:00
migration.mdx Enable doc in Spanish (#16518) 2022-04-04 10:25:46 -04:00
model_sharing.mdx Fix PushToHubCallback import in Share a model docs (#21457) 2023-02-06 09:26:22 -05:00
model_summary.mdx Refactor model summary (#21408) 2023-02-15 10:35:14 -08:00
multilingual.mdx Enable doc in Spanish (#16518) 2022-04-04 10:25:46 -04:00
notebooks.md Enable doc in Spanish (#16518) 2022-04-04 10:25:46 -04:00
pad_truncation.mdx Enable doc in Spanish (#16518) 2022-04-04 10:25:46 -04:00
perf_hardware.mdx [WIP] [doc] performance/scalability revamp (#15723) 2022-05-16 13:36:41 +02:00
perf_infer_cpu.mdx add doc for (#20525) 2022-12-01 16:52:13 +01:00
perf_infer_gpu_many.mdx add doc for (#20525) 2022-12-01 16:52:13 +01:00
perf_infer_gpu_one.mdx [Doc] Fix int8 docs (#21487) 2023-02-07 15:09:27 +01:00
perf_infer_special.mdx Improve performance docs (#17750) 2022-06-23 14:51:54 +02:00
perf_train_cpu_many.mdx update cpu related doc (#20444) 2022-11-28 08:54:35 -05:00
perf_train_cpu.mdx Add perf numbers for perf_train_cpu (#20974) 2023-02-06 09:20:43 -05:00
perf_train_gpu_many.mdx Fix Typo in Docs for GPU (#20509) 2022-11-30 10:41:18 -05:00
perf_train_gpu_one.mdx Migrate torchdynamo to torch.compile (#20634) 2022-12-08 11:18:52 -05:00
perf_train_special.mdx Fix Typo in Docs for GPU (#20509) 2022-11-30 10:41:18 -05:00
perf_train_tpu_tf.mdx Typos/fixes to link syntax (#21450) 2023-02-07 15:19:19 +00:00
perf_train_tpu.mdx Fix Typo in Docs for GPU (#20509) 2022-11-30 10:41:18 -05:00
performance.mdx Fix Typo in Docs for GPU (#20509) 2022-11-30 10:41:18 -05:00
perplexity.mdx Fix incorrect size of input for 1st strided window length in Perplexity of fixed-length models (#18906) 2022-09-06 15:20:12 -04:00
philosophy.mdx Update doc examples feature extractor -> image processor (#20501) 2022-11-30 14:50:55 +00:00
pipeline_tutorial.mdx [pipeline] A simple fix for half-precision & 8bit models (#21479) 2023-02-10 10:26:17 +01:00
pipeline_webserver.mdx Update quality tooling for formatting (#21480) 2023-02-06 18:10:56 -05:00
pr_checks.mdx Cleanup quality (#21493) 2023-02-07 12:27:31 -05:00
preprocessing.mdx Updates to computer vision section of the Preprocess doc (#21181) 2023-01-19 08:43:36 -05:00
quicktour.mdx Clarify available pipelines in quicktour (#21607) 2023-02-13 11:37:48 -08:00
run_scripts.mdx Just re-reading the whole doc every couple of months 😬 (#18489) 2022-08-06 09:38:55 +02:00
sagemaker.mdx Enable doc in Spanish (#16518) 2022-04-04 10:25:46 -04:00
serialization.mdx Add X-MOD (#20939) 2023-02-10 15:32:06 +01:00
task_summary.mdx Remove trailing 'extractive' word from en documentation (#21594) 2023-02-13 10:09:00 -05:00
tasks_explained.mdx Update task summary (#21067) 2023-02-02 11:41:27 -08:00
testing.mdx fixed spelling error in testing.mdx (#20220) 2022-11-15 09:40:06 -05:00
tf_xla.mdx Rewrite a couple of lines in the TF XLA doc (#21177) 2023-01-18 17:53:05 +00:00
tokenizer_summary.mdx Update tokenizer_summary.mdx (#20135) 2022-11-15 01:18:13 +01:00
torchscript.mdx Breakup export guide (#19271) 2022-10-03 13:18:29 -07:00
training.mdx Fix code example in training tutorial (#21201) 2023-01-20 07:38:15 -08:00
troubleshooting.mdx Update doc examples feature extractor -> image processor (#20501) 2022-11-30 14:50:55 +00:00