amyeroberts
3412f5979d
Use PyAV instead of Decord in examples ( #21572 )
...
* Use PyAV instead of Decord
* Get frame indices
* Fix number of frames
* Update src/transformers/models/videomae/image_processing_videomae.py
* Fix up
* Fix copies
* Update timesformer doctests
* Update docstrings
2023-03-02 12:30:38 +00:00
Alara Dirik
269b054939
Add ALIGN to transformers ( #21741 )
...
Adds the ALIGN model to transformers. ALIGN is introduced in "Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision" by Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc V. Le, Yunhsuan Sung, Zhen Li, Tom Duerig.
2023-03-01 21:23:31 +03:00
Matt
f7c618e3b0
Add TFVisionTextDualEncoder ( #21873 )
...
* Temporary commit to stash everything so far
* Temporary commit to stash everything so far
* stash commit
* Refactor from_pretrained
* Fix final test, make fixup
* Update dummies
* Add model to TEST_FILES_WITH_NO_COMMON_TESTS
* Update src/transformers/models/vision_text_dual_encoder/modeling_tf_vision_text_dual_encoder.py
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
* Update src/transformers/models/vision_text_dual_encoder/modeling_tf_vision_text_dual_encoder.py
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
* Update src/transformers/models/vision_text_dual_encoder/modeling_tf_vision_text_dual_encoder.py
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
* Update src/transformers/models/vision_text_dual_encoder/modeling_tf_vision_text_dual_encoder.py
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
* Add TFVisionTextDualEncoder to utils/documentation_tests.txt
* make fixup
---------
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
2023-03-01 18:00:48 +00:00
Yih-Dar
53735d7c3b
Add an utility file to get information from test files ( #21856 )
...
* Add an utility file to get information from test files
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-03-01 17:53:29 +01:00
Yih-Dar
871c31a6f1
🔥 Rework pipeline testing by removing PipelineTestCaseMeta 🚀 ( #21516 )
...
* Add PipelineTesterMixin
* remove class PipelineTestCaseMeta
* move validate_test_components
* Add for ViT
* Add to SPECIAL_MODULE_TO_TEST_MAP
* style and quality
* Add feature-extraction
* update
* raise instead of skip
* add tiny_model_summary.json
* more explicit
* skip tasks not in mapping
* add availability check
* Add Copyright
* A way to diable irrelevant tests
* update with main
* remove disable_irrelevant_tests
* skip tests
* better skip message
* better skip message
* Add all pipeline task tests
* revert
* Import PipelineTesterMixin
* subclass test classes with PipelineTesterMixin
* Add pipieline_model_mapping
* Fix import after adding pipieline_model_mapping
* Fix style and quality after adding pipieline_model_mapping
* Fix one more import after adding pipieline_model_mapping
* Fix style and quality after adding pipieline_model_mapping
* Fix test issues
* Fix import requirements
* Fix mapping for MobileViTModelTest
* Update
* Better skip message
* pipieline_model_mapping could not be None
* Remove some PipelineTesterMixin
* Fix typo
* revert tests_fetcher.py
* update
* rename
* revert
* Remove PipelineTestCaseMeta from ZeroShotAudioClassificationPipelineTests
* style and quality
* test fetcher for all pipeline/model tests
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-02-28 19:40:57 +01:00
Yih-Dar
aab895c396
Make Slack CI reporting stronger ( #21823 )
...
* Use token
* Avoid failure
* better error
* Fix
* fix style
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-02-28 17:12:44 +01:00
Yih-Dar
354b338316
Remove gptsan_japanese from doctest list to avoid GPU OOM ( #21722 )
...
remove from doctest list to avoid GPU OOM
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-02-22 10:51:00 +01:00
Aaron Gokaslan
5e8c8eb5ba
Apply ruff flake8-comprehensions ( #21694 )
2023-02-22 09:14:54 +01:00
Yih-Dar
ed6ceb7649
Fix typo in PROCESSOR_MAPPING_NAMES and add tests ( #21703 )
...
* Add test
* Fix GITProcessor
* Update
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-02-21 09:38:26 +01:00
tanreinama
f56174ac5b
add GPTSAN model (reopen) ( #21291 )
...
* add GPTSAN-Japanese
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN (update for review)
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* fix typo in comment text
* add GPTSAN
* add GPTSAN
* add GPTSAN
* add GPTSAN
* fix document and comments
* fix class name GPTSAN->GPTSan
* fix import and test for tokenizer
2023-02-20 11:25:27 +01:00
Andy Ehrenberg
2840272c5f
add flax whisper implementation ( #20479 )
...
* add flax whisper implementation
* rever change to setup
* remove unused imports
* revert generation changes
* flax whisper docs
* docs
* import order
* import sorting
* isort
* add dummy objects
* doc formatting
* formatting
* remove trailing whitespaces
* fix flax whisper docs
* add generation logic to unlock flax whisper
* remove scans
* give credits to Flax Bart implementation
* remove unused imports
* add license
* remove assert
* more credits to Bart
* fix style
* formatting
* support left padding
* add flax whisper generation test
* remove copied from comments whenever not a full copy
* fix docstrings for logits processors
* revert change to FlaxForceTokensLogitsProcessor
* revert doc changes
* improve generation docs
* reorganize
* formatting
* cleanup docs
* add tests
* handle empty list case
* fix forced decoder ids in flax tests
* add flax whisper to inits
* upate dummy objects
* docs for FlaxAutoModelForSpeechSeq2Seq
* fix decoder_position_ids computation in pretrained model decode/__call__ fns
* add Copied from statements as necessary
* compute position_ids only in __call__ and decode methods of pretrained model subclasses
* improve readabilityof compute positional embeddings
* check dimensionality of input_features instead of hidden_states
* copied from statement for init_cache
* formatting
* fix copies
* fix copies
* pass attention mask to encoder layers
* fix decoder module outputs
* set dtype
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
* smaller flax model for whisper test
* Update src/transformers/generation/flax_utils.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/models/whisper/modeling_flax_whisper.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update tests/models/whisper/test_modeling_flax_whisper.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* cleanup
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/models/whisper/modeling_flax_whisper.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* bias cleanup
* doc fix
* align style for force tokens processor
* readability
* fix input shape in tests
* revert FlaxGenerationMixin docstring
* formatting
* fix tests
* fix imports
* consistent encoder hidden states
* consistent hidden states
* input shapes
* typo
* partial class trick
* partial class for input shape
* base_class with correct input shape
* partial base classes
* match by name
* set main_input_name
* compare on names
* formatting
* remove unused import
* safer position ids computation
* safer position id computation
* Update src/transformers/models/whisper/modeling_flax_whisper.py
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
* Update src/transformers/models/whisper/modeling_flax_whisper.py
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
* remove identical inherited tests
* fix prompt ids in tests
* use generation config
* use jnp array
* better var names
* more explicit bias use
* import transformers
* formatting
* test formatting
* remove unused imports
* remove unused imports
* formatting
* isort
* docs
* fix ln orders for encoder hidden states
* whisper unique generation stuff
* flake
* use finfo for attention bias
* docs
* Update src/transformers/generation/flax_utils.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* docs
* add timestamp flax test
* jit for timestamps
* formatting
* clean up timestamps processor
* formatting
* remove if_true
* cleanup
---------
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
2023-02-20 09:17:40 +01:00
Arthur
c236a62172
[CLAP] Add CLAP to the library ( #21370 )
...
* add model like clip
* update
* text model ok
* clap text works
* some refactor
- `CLAPVision` to `CLAPAudio`
- refactor kwargs of audio modules
* more refactor
* more refactor
* more refactor
* correct fusion
* more refactor
* new modules
* add basic processor
* fixup
* remove whisper copioed from
* audio logits match
* add doc
* correct filters mel and add maxlength
* style
* few fixes
* forward passes
* fixup
* fixup
* some clean up
* remove mels form the dictionnary
* pad after the repeat
* update padding when dsmaller
* fix padding
* style
* use swin patch merging
* use copied from swin
* processor with any tokenizer
* more copied from
* some clean up
* more refactor
* fix mel when rand_trunc
* style
* remove unused imports
* update processing
* remove image processing tests
* add testing fiel
* fixmodeling issues
* replace with `is_longer`
* clap in serialization
* more refactor
* `make fixup`
* make fixup
* fix feature extractor
* update test feature extractor
* `make fixup`
* clean up config
* more clean up
* more cleanup
* update tests
* refactor tests and inits
* removeCLAP vision config
* remove CLAP from image procssing auto and dummy vision objects
* update inits
* style
* re order classes in modeling clap
* Use roberta tokenizer as the other weights are not open sourced
* small cleaup
* remove tokenization CLAP
* processor tokenizr is roberta
* update feature extraction doc
* remove vclap from model zero shot
* update f_min and f_max to frequency_xx
* some changes
- fix modeling keys
- add `is_longer` in the forward pass
- make fixup
* make fixup
* consistent behavior ebtween rand_crop and fusion
* add numpy resize and bilinear and documentation
* move resizing to image utils
* clean feature extraction
* import resize from correct file
* resize in image transforms
* update
* style
* style
* nit
* remove unused arguments form the feature extractor
* style
* few fixes + make fixup
* oops
* fix more tests
* add zero shot audio classification pipeline
* update zeroshot classification pipeline
* fixup
* fix copies
* all CI tests pass
* make fixup + fix docs
* fix docs
* fix docs
* update tests pip;eline
* update zero shot pipeline
* update feature extraction clap
* update tokenization auto
* use nested simplify
* update pipeline tests
* Apply suggestions from code review
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* split in two lines
* fixes
* refactor
* clean up
* add integration tests
* update config docstring
* style
* update processor
* fix processor test
* fix feat extractor tests
* update docs
* Apply suggestions from code review
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* fix readmes
* fix tips
* Update src/transformers/models/auto/configuration_auto.py
* update doc and remove todo -> properly explained
* fix idx and typo
* typoe
* cleanup config
* cleanup tests, styles and doc
* ignore docstyle on image transform
* add conversion script
* remove the `clap` indx in favor of `CLAP`
* update __init
* nits
* Update src/transformers/pipelines/__init__.py
* fix bug
* clarifiy config
* fix copy
* fix init
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* fix model output
* fix comment
* make fixup
* make fixup
* rename to `Clap`
* replace to `Clap`
* replace to `Clap`
* repo consistency
* again repo-consistency
* make fixup
* Apply suggestions from code review
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
* add config
* changes
* update conversion
* Apply suggestions from code review
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
* remove unused function
* update based on code reviews
* style
* more comments
* cleanup
* clean up
* style
* apply suggestions
* Empty commit
* pipeline will be added in a different PR
* update calls to audio utils functions
* update pipeline init
* style
* style
* styling again
* use pad
* fix repo-consistency
* update utils and add doc for audio utils
* clean up resize by using torch. update inits accordingly
* style
* CLap's tokenizer is RobertA
* add audio utils to internal toctreee
* update totctree
* style
* update documentation and normalize naming accross audio utils and feature extraction clap
* style
* clean up
* update doc and typos
* fix doctest
* update modelin code, got rid of a lot of reshaping
* style on added doc audio utils
* update modeling clap
* style
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* docstringvariables with CLAP
* rename key
* update modeling CLAP
* update audio utils docstring
* update processing clap
* fix readmes
* fix toctree
* udpate configuration clap
* fix init
* make fixup
* fix
* fix
* update naming
* update
* update checkpoint path
* Apply suggestions from code review
* Major refactoring
* Update src/transformers/models/clap/configuration_clap.py
* merge
---------
Co-authored-by: younesbelkada <younesbelkada@gmail.com>
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
2023-02-16 20:59:27 +01:00
Connor Henderson
0f96c26de6
refactor: Make direct_transformers_import util ( #21652 )
...
* refactor: Make direct_import util
* edit direct import fn
* add docstring
* make import function specific to transformers only
* edit doc string
2023-02-16 11:32:32 -05:00
Sylvain Gugger
9d1116e995
Update deprecated load_module ( #21651 )
2023-02-15 15:57:24 -05:00
Zineng Tang
a0e69a9375
Add TVLT ( #20725 )
...
* Update image_processing_tvlt.py
* Update modeling_tvlt.py
* Update
* Update modeling_tvlt.py
* Create tvlt.mdx
* Update configuration_tvlt.py
* Update modeling_tvlt.py
* Update test_modeling_tvlt.py
* Update modeling_tvlt.py
* Update modeling_tvlt.py
* Update image_processing_tvlt.py
* Update feature_extraction_tvlt.py
* Update tvlt models
* Update tests
* Update
* Update
* Update tests
* Update README_ko.md
* Update README_ja.md
* Update README_ko.md
* Update README_zh-hans.md
* Update docs/source/en/model_doc/tvlt.mdx
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update docs/source/en/model_doc/tvlt.mdx
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/tvlt/configuration_tvlt.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/tvlt/configuration_tvlt.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/tvlt/configuration_tvlt.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/tvlt/configuration_tvlt.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update tvlt.mdx
* Update modeling_tvlt.py
* Update configuration_tvlt.py
* Update modeling_tvlt.py
* Update modeling_tvlt.py
* Update modeling_tvlt.py
* Update modeling_tvlt.py
* Add files via upload
* Update model
* Update modeling_tvlt.py
* Update tvlt models
* Update src/transformers/models/tvlt/__init__.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/tvlt/__init__.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/tvlt/image_processing_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/tvlt/image_processing_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Add files via upload
* Add files via upload
* Delete modeling_tvlt.py
* Delete feature_extraction_tvlt.py
* Delete configuration_tvlt.py
* Delete image_processing_tvlt.py
* Delete processing_tvlt.py
* Update tvlt
* Update src/transformers/models/tvlt/configuration_tvlt.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/models/tvlt/image_processing_tvlt.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* Update src/transformers/models/tvlt/configuration_tvlt.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* Update tests/models/tvlt/test_modeling_tvlt.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update tests/models/tvlt/test_modeling_tvlt.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/models/tvlt/modeling_tvlt.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/models/tvlt/modeling_tvlt.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update README.md
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update README_es.md
* Update README_hd.md
* Update README_ja.md
* Update README_ko.md
* Update README_zh-hans.md
* Update README_zh-hant.md
* Update index.mdx
* Update tvlt.mdx
* Update tvlt.mdx
* Update configuration_tvlt.py
* Update src/transformers/models/tvlt/image_processing_tvlt.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/models/tvlt/image_processing_tvlt.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/models/tvlt/image_processing_tvlt.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/models/tvlt/image_processing_tvlt.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/models/tvlt/modeling_tvlt.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update modeling_tvlt.py
* Add files via upload
* Update tvlt.mdx
* Update modeling_auto.py
* Add files via upload
* Add files via upload
* Update dummy_pt_objects.py
* Update __init__.py
* Update feature_extraction_tvlt.py
* Update feature_extraction_tvlt.py
* Update image_processing_tvlt.py
* Update modeling_auto.py
* Update test_feature_extraction_tvlt.py
* Update test_processor_tvlt.py
* Update test_feature_extraction_tvlt.py
* Add files via upload
* Update test_image_processor_tvlt.py
* Update tests/models/tvlt/test_processor_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/tvlt/modeling_tvlt.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/tvlt/processing_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/tvlt/modeling_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/tvlt/modeling_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update tests/models/tvlt/test_image_processor_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/tvlt/modeling_tvlt.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update tests/models/tvlt/test_image_processor_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/tvlt/modeling_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update tests/models/tvlt/test_image_processor_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update tests/models/tvlt/test_image_processor_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update tests/models/tvlt/test_modeling_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/tvlt/modeling_tvlt.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/tvlt/modeling_tvlt.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/tvlt/configuration_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update tests/models/tvlt/test_feature_extraction_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/tvlt/processing_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update docs/source/en/model_doc/tvlt.mdx
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/tvlt/configuration_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/tvlt/configuration_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/tvlt/configuration_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/tvlt/configuration_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/tvlt/configuration_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/tvlt/configuration_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/tvlt/configuration_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/tvlt/configuration_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/tvlt/configuration_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/tvlt/configuration_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/tvlt/feature_extraction_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/tvlt/configuration_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/tvlt/feature_extraction_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/tvlt/feature_extraction_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/tvlt/feature_extraction_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update feature_extraction_tvlt.py
* Update feature_extraction_tvlt.py
* Update src/transformers/models/tvlt/image_processing_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/tvlt/image_processing_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update image_processing_tvlt.py
* Update src/transformers/models/tvlt/modeling_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update modeling_tvlt.py
* Update modeling_tvlt.py
* Update modeling_tvlt.py
* Update test_image_processor_tvlt.py
* Update tests/models/tvlt/test_modeling_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update tests/models/tvlt/test_modeling_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update tests/models/tvlt/test_modeling_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update tests/models/tvlt/test_modeling_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update tests/models/tvlt/test_modeling_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update tests/models/tvlt/test_modeling_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update tests/models/tvlt/test_modeling_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update tests/models/tvlt/test_modeling_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update tests/models/tvlt/test_modeling_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Add files via upload
* Add files via upload
* Update modeling_tvlt.py
* Update modeling_tvlt.py
* Update modeling_tvlt.py
* Add files via upload
* Update docs/source/en/model_doc/tvlt.mdx
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update image_processing_tvlt.py
* Add files via upload
* Add files via upload
* Update tvlt.mdx
* Update docs/source/en/model_doc/tvlt.mdx
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update docs/source/en/model_doc/tvlt.mdx
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/tvlt/image_processing_tvlt.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/tvlt/image_processing_tvlt.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update docs/source/en/model_doc/tvlt.mdx
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
* Update docs/source/en/model_doc/tvlt.mdx
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
* Update src/transformers/models/tvlt/configuration_tvlt.py
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
* Add files via upload
* Add files via upload
* Add files via upload
* Add files via upload
* Update modeling_auto.py
* Update tvlt.mdx
* Update dummy_pt_objects.py
* Update feature_extraction_tvlt.py
* Update modeling_tvlt.py
* Update test_feature_extraction_tvlt.py
* Update test_image_processor_tvlt.py
* Update test_feature_extraction_tvlt.py
* Update modeling_tvlt.py
* Update dummy_pt_objects.py
* Update dummy_speech_objects.py
* Add files via upload
* Update README_hd.md
* Update modeling_tvlt.py
* Update modeling_tvlt.py
* Update modeling_tvlt.py
* Update modeling_tvlt.py
* Update modeling_tvlt.py
* Update modeling_tvlt.py
* Update test_modeling_tvlt.py
* Update src/transformers/models/tvlt/configuration_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/tvlt/feature_extraction_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/tvlt/image_processing_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/tvlt/image_processing_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/tvlt/modeling_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/tvlt/image_processing_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/tvlt/modeling_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update MAE processing
* Update modeling_tvlt.py
* Update modeling_tvlt.py
* Update modeling
* Update style
* Update src/transformers/models/tvlt/modeling_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/tvlt/modeling_tvlt.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update check_repo.py
* Update tvlt.mdx
* Update __init__.py
* Update tests
* Update tvlt models
* Update configuration_tvlt.py
* Update configuration_tvlt.py
* Update image_processing_tvlt.py
* Update dummy_pt_objects.py
* Add files via upload
* Update test_modeling_tvlt.py
* Update test_feature_extraction_tvlt.py
* Update test_feature_extraction_tvlt.py
* Update test_feature_extraction_tvlt.py
* Update test_feature_extraction_tvlt.py
* Update test_feature_extraction_tvlt.py
* Update test_feature_extraction_tvlt.py
---------
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
2023-02-15 18:10:30 +00:00
Susnato Dhar
0c9c8472e6
Add Ernie-M Model to huggingface ( #21349 )
...
* config and tokenization(fast too) changed and ErnieEncoder added
* Slow Tokenization Added
* Tokenizer(slow) is now working and Fast Tokenizer removed
* Added Config code
* Added Base Model and utils
* ErnieMModel is now working
* All added except tests
* All tests passed except ErnieUIEM
* All tests passed
* all fixes done
* all fixes done
* fixed MAP
* fixed check_code_quality
* fixed Build PR Documentation issue
* Added changes(comments) and also updated to the latest upstream/main
* Added fixup
* Added # Copied comments
* Added fixup
* Added more comments and some nits
* Added fixup
* Fixed README_hd.md
* Added more fixes
* ErnieMTokenizer (being sentencepiece) protected and other docs edited
* Added code_quality fix
* Fixed for
* Added more fix
* modified AZ
* ernie-m tokenization test added!
* attention mask part fixed(with 0->self.config.pad_token_id)
* applied make fixup
2023-02-15 09:24:56 -05:00
Maria Khalusova
3baa407f92
Add: document question answering task guide ( #21518 )
...
* document question answering guide
* Added the list of supported models
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* switched to AutoProcessor
* feedback addressed
* Apply suggestions from code review
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
* Update docs/source/en/tasks/document_question_answering.mdx
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
* more feedback addressed
* addressed comments about evaluation loss
* added appropriate image link
* make style
* typo fix
* resolving toc conflict
* fixed the image link
---------
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
2023-02-13 09:24:56 -05:00
Yih-Dar
b47a16743b
Remove more unused attributes in config classes ( #21543 )
...
* Remove unused decoder_layerdrop
* Update SPECIAL_CASES_TO_ALLOW for MT5Config
* Remove unused position_embedding_init_scale
* Remove unused decoder_max_relative_position
* Use unused decoder_max_relative_position
* Remove unused init_std
* Remove unused forgotten attributes
* Remove unused patch_norm
* Remove unused max_seq_len
* Update SPECIAL_CASES_TO_ALLOW for OneFormerConfig
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-02-10 22:57:28 +01:00
Han Wu
862e8e4f4a
Added timesformer configuration ( #21446 )
...
* Added timesformer configuration
Co-authored-by: JuheonChu <chuj@dickinson.edu>
* Create documentation_tests.txt
* Update documentation_tests.txt
Co-authored-by: JuheonChu <chuj@dickinson.edu>
* Delete documentation_tests.txt
Updates, Deleting "src/transformers/utils/documentation_tests.txt" file.
Co-authored-by: JuheonChu <chuj@dickinson.edu>
* Create documentation_tests.txt
Co-authored-by: JuheonChu <chuj@dickinson.edu>
* Delete documentation_tests.txt
Co-authored-by: JuheonChu <chuj@dickinson.edu>
---------
Co-authored-by: JuheonChu <chuj@dickinson.edu>
2023-02-10 22:54:40 +01:00
Yih-Dar
4f831e661b
Goodbye to Blip-2 doctests ( #21566 )
...
Byebye Blip-2 doctest
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-02-10 18:37:06 +01:00
Jannis Vamvas
b0d539ccad
Add X-MOD ( #20939 )
...
* Add X-MOD to Readme
* Add documentation for X-MOD
* Implement X-MOD
* Fix formatting of X-MOD docs
* Change signature of X-MOD forward methods to use lang_ids
* Minor changes
* Rebase with main and run make fix-copies
* Make suggested changes to docstrings
* Improve code readability
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
* Fix code style
* Conversion script: Remove asserts and type annotations
* Remove _TOKENIZER_FOR_DOC
* XMOD -> Xmod
* Update copyright note
* Fix doctests
* Fix docstring
* Add integration test for FillMaskPipeline
* Revert "Add integration test for FillMaskPipeline"
This reverts commit 4381eb3b1d0f5d85785f89caba83928e6efa6d1f.
* Add end-to-end integration test for mask fill
* make style
* Rebase with main and make fix-copies
---------
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
2023-02-10 15:32:06 +01:00
NielsRogge
d7f1e7c009
Add BLIP-2 ( #21441 )
...
* First draft
* More improvements
* More improvements
* Improve conversion script
* Convert all weights
* Make forward pass work
* Make logits match
* More improvements
* More improvements
* More improvements
* Use get_input_embeddings
* Improve some more
* Improve model tests
* Improve model tests
* More improvements
* Fix processor
* Update files
* Update prepare_inputs_for_generation
* More improvements
* Fix copies
* More fixes
* Make fixup
* More improvements
* Add support for seq2seq language model
* More improvements
* Fix test
* More improvements
* Improve conversion script
* Remove some todo's
* Fix README's
* Improve conversion script
* Fix generation
* Fix style and remove Blip2Model
* Fix model outputs
* More improvements
* Set eos_token_id in config
* Fix quality
* Small improvements
* Add processor tests
* More improvements
* Apply suggestions
* Apply suggestions
* Add integration test
* Update image URL
* Add integration test
* Fix model_type
* Update style
* Improve docs
* Add doc tests
* Fix copies
* Remove tests which are passing
* Improve some more
* Add tests for seq2seq language models
* Minor fix
* Convert more checkpoints
* finalize CI
* Fix blip and blip2 processors
* add `accelerate` support for `blip2`
* clean up
* make style
* Update conversion script
* Update conversion script some more
* Update organization
* revert toc file
* add blip-2 to toc file
* Some more improvements
* Fix docstring
* Improve docs
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: younesbelkada <younesbelkada@gmail.com>
2023-02-09 16:52:11 +01:00
Sylvain Gugger
67d074874d
Cleanup quality ( #21493 )
...
* Remove mentions of flake8/isort
* Clean up inits
* Deall with all other inits
* Last special rule for dummy files
2023-02-07 12:27:31 -05:00
Yih-Dar
479322bfaa
A new test to check config attributes being used ( #21453 )
...
* Add a new test to check config attributes being used
* Add a new test to check config attributes being used
* Add a new test to check config attributes being used
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Apply suggestions
* Update allowed cases - part 1
* Update allowed cases - part 2
* final
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-02-07 17:49:30 +01:00
Sylvain Gugger
6f79d26442
Update quality tooling for formatting ( #21480 )
...
* Result of black 23.1
* Update target to Python 3.7
* Switch flake8 to ruff
* Configure isort
* Configure isort
* Apply isort with line limit
* Put the right black version
* adapt black in check copies
* Fix copies
2023-02-06 18:10:56 -05:00
Darren Tuit
6c62cfb2ef
exclude deleted files in the fixup script ( #21436 )
...
exclude deleted files from fixup script
2023-02-03 12:57:02 -05:00
Matthijs Hollemans
e4bacf6614
[WIP] add SpeechT5 model ( #18922 )
...
* make SpeechT5 model by copying Wav2Vec2
* add paper to docs
* whoops added docs in wrong file
* remove SpeechT5Tokenizer + put CTC back in the name
* remove deprecated class
* remove unused docstring
* delete SpeechT5FeatureExtractor, use Wav2Vec2FeatureExtractor instead
* remove classes we don't need right now
* initial stab at speech encoder prenet
* add more speech encoder prenet stuff
* improve SpeechEncoderPrenet
* add encoder (not finished yet)
* add relative position bias to self-attention
* add encoder CTC layers
* fix formatting
* add decoder from BART, doesn't work yet
* make it work with generate loop
* wrap the encoder into a speech encoder class
* wrap the decoder in a text decoder class
* changed my mind
* changed my mind again ;-)
* load decoder weights, make it work
* add weights for text decoder postnet
* add SpeechT5ForCTC model that uses only the encoder
* clean up EncoderLayer and DecoderLayer
* implement _init_weights in SpeechT5PreTrainedModel
* cleanup config + Encoder and Decoder
* add head + cross attention masks
* improve doc comments
* fixup
* more cleanup
* more fixup
* TextDecoderPrenet works now, thanks Kendall
* add CTC loss
* add placeholders for other pre/postnets
* add type annotation
* fix freeze_feature_encoder
* set padding tokens to 0 in decoder attention mask
* encoder attention mask downsampling
* remove features_pen calculation
* disable the padding tokens thing again
* fixup
* more fixup
* code review fixes
* rename encoder/decoder wrapper classes
* allow checkpoints to be loaded into SpeechT5Model
* put encoder into wrapper for CTC model
* clean up conversion script
* add encoder for TTS model
* add speech decoder prenet
* add speech decoder post-net
* attempt to reconstruct the generation loop
* add speech generation loop
* clean up generate_speech
* small tweaks
* fix forward pass
* enable always dropout on speech decoder prenet
* sort declaration
* rename models
* fixup
* fix copies
* more fixup
* make consistency checker happy
* add Seq2SeqSpectrogramOutput class
* doc comments
* quick note about loss and labels
* add HiFi-GAN implementation (from Speech2Speech PR)
* rename file
* add vocoder to TTS model
* improve vocoder
* working on tokenizer
* more better tokenizer
* add CTC tokenizer
* fix decode and batch_code in CTC tokenizer
* fix processor
* two processors and feature extractors
* use SpeechT5WaveformFeatureExtractor instead of Wav2Vec2
* cleanup
* more cleanup
* even more fixup
* notebooks
* fix log-mel spectrograms
* support reduction factor
* fixup
* shift spectrograms to right to create decoder inputs
* return correct labels
* add labels for stop token prediction
* fix doc comments
* fixup
* remove SpeechT5ForPreTraining
* more fixup
* update copyright headers
* add usage examples
* add SpeechT5ProcessorForCTC
* fixup
* push unofficial checkpoints to hub
* initial version of tokenizer unit tests
* add slow test
* fix failing tests
* tests for CTC tokenizer
* finish CTC tokenizer tests
* processor tests
* initial test for feature extractors
* tests for spectrogram feature extractor
* fixup
* more fixup
* add decorators
* require speech for tests
* modeling tests
* more tests for ASR model
* fix imports
* add fake tests for the other models
* fixup
* remove jupyter notebooks
* add missing SpeechT5Model tests
* add missing tests for SpeechT5ForCTC
* add missing tests for SpeechT5ForTextToSpeech
* sort tests by name
* fix Hi-Fi GAN tests
* fixup
* add speech-to-speech model
* refactor duplicate speech generation code
* add processor for SpeechToSpeech model
* add usage example
* add tests for speech-to-speech model
* fixup
* enable gradient checkpointing for SpeechT5FeatureEncoder
* code review
* push_to_hub now takes repo_id
* improve doc comments for HiFi-GAN config
* add missing test
* add integration tests
* make number of layers in speech decoder prenet configurable
* rename variable
* rename variables
* add auto classes for TTS and S2S
* REMOVE CTC!!!
* S2S processor does not support save/load_pretrained
* fixup
* these models are now in an auto mapping
* fix doc links
* rename HiFiGAN to HifiGan, remove separate config file
* REMOVE auto classes
* there can be only one
* fixup
* replace assert
* reformat
* feature extractor can process input and target at same time
* update checkpoint names
* fix commit hash
2023-02-03 12:43:46 -05:00
amyeroberts
e5db7051a8
Add TF image classification example script ( #19956 )
...
* TF image classification script
* Update requirements
* Fix up
* Add tests
* Update test fetcher
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Fix directory path
* Adding `zero-shot-object-detection` pipeline doctest. (#20274 )
* Adding `zero-shot-object-detection` pipeline doctest.
* Remove nested_simplify.
* Add generate kwargs to `AutomaticSpeechRecognitionPipeline` (#20952 )
* Add generate kwargs to AutomaticSpeechRecognitionPipeline
* Add test for generation kwargs
* Trigger CI
* Data collator returns np
* Update feature extractor -> image processor
* Bug fixes - updates to reflect changes in API
* Update flags to match PT & run faster
* Update instructions - Maria's comment
* Update examples/tensorflow/image-classification/README.md
* Remove slow decorator
---------
Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com>
Co-authored-by: bofeng huang <bofenghuang7@gmail.com>
Co-authored-by: Sylvain Gugger <Sylvain.gugger@gmail.com>
2023-02-01 19:09:36 +00:00
Joao Gante
623346ab18
Template for framework-agnostic tests ( #21348 )
2023-01-31 11:33:18 +00:00
NielsRogge
5451f8896c
Add DETA ( #20983 )
...
* First draft
* Add initial draft of conversion script
* Convert all weights
* Fix config
* Add image processor
* Fix DetaImageProcessor
* Run make fix copies
* Remove timm dependency
* Fix dummy objects
* Improve loss function
* Remove conv_encoder attribute
* Update conversion scripts
* Improve postprocessing + docs
* Fix copied from statements
* Add tests
* Improve postprocessing
* Improve postprocessing
* Update READMEs
* More improvements
* Fix rebase
* Add is_torchvision_available
* Add torchvision dependency
* Fix typo and README
* Fix bug
* Add copied from
* Fix style
* Apply suggestions
* Fix thanks to @ydshieh
* Fix another dependency check
* Simplify image processor
* Add scipy
* Improve code
* Add threshold argument
* Fix bug
* Set default threshold
* Improve integration test
* Add another integration test
* Update setup.py
* Address review
* Improve deformable attention function
* Improve copied from
* Use relative imports
* Address review
* Replace assertions
* Address review
* Update dummies
* Remove dummies
* Address comments, update READMEs
* Remove custom kernel code
* Add image processor tests
* Add requires_backends
* Add minor comment
* Update scripts
* Update organization name
* Fix defaults, add doc tests
* Add id2label for object 365
* Fix tests
* Update task guide
2023-01-31 10:43:10 +01:00
Yih-Dar
c749bd405e
Pipeline testing - using tiny models on Hub ( #20426 )
...
* rework pipeline tests
* run pipeline tests
* fix
* fix
* fix
* revert the changes in get_test_pipeline() parameter list
* fix expected error message
* skip a test
* clean up
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-01-30 10:39:43 +01:00
Maria Khalusova
73a2ff6974
Automated compatible models list for task guides ( #21338 )
...
* initial commit. added tip placeholders and a script
* removed unused imports, fixed paths
* fixed generated links
* make style
* split language modeling doc into two: causal language modeling and masked language modeling
* added check_task_guides.py to make fix-copies
* review feedback addressed
2023-01-27 13:19:28 -05:00
Yih-Dar
857bad6e53
check paths in utils/documentation_tests.txt ( #21315 )
...
* check paths in utils/documentation_tests.txt
* check paths in utils/documentation_tests.txt
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-01-26 15:33:47 +01:00
Yih-Dar
31336dcf3f
Fix 2 paths in the doctest list ( #21314 )
...
fix the list
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-01-26 12:07:08 +01:00
Anahita Bhiwandiwalla
3a6e4a221c
Add BridgeTower model ( #20775 )
...
* Commit with BTModel and latest HF code
* Placeholder classes for BTForMLM and BTForITR
* Importing Bert classes from transformers
* Removed objectives.py and dist_utils.py
* Removed swin_transformer.py
* Add image normalization, BridgeTowerForImageAndTextRetrieval
* Add center_crop
* Removing bert tokenizer and LCI references
* Tested config loading from HF transformers hub
* Removed state_dict updates and added path to hub
* Enable center crop
* Getting image_size from config, renaming num_heads and num_layers
* Handling max_length in BridgeTowerProcessor
* Add BridgeTowerForMaskedLM
* Add doc string for BridgeTowerConfig
* Add doc strings for BT config, processor, image processor
* Adding docs, removed swin
* Removed convert_bridgetower_original_to_pytorch.py
* Added doc files for bridgetower, removed is_vision
* Add support attention_mask=None and BridgeTowerModelOutput
* Fix formatting
* Fixes with 'make style', 'make quality', 'make fixup'
* Remove downstream tasks from BridgeTowerModel
* Formatting fixes, add return_dict to BT models
* Clean up after doc_test
* Update BTModelOutput return type, fix todo in doc
* Remove loss_names from init
* implement tests and update tuples returned by models
* Add image reference to bridgetower.mdx
* after make fix-copies, make fixup, make style, make quality, make repo-consistency
* Rename class names with BridgeTower prefix
* Fix for image_size in BTImageProcessor
* implement feature extraction bridgetower tests
* Update image_mean and image_std to be list
* remove unused import
* Removed old comments
* Rework CLIP
* update config in tests followed config update
* Formatting fixes
* Add copied from for BridgeTowerPredictionHeadTransform
* Update bridgetower.mdx
* Update test_feature_extraction_bridgetower.py
* Update bridgetower.mdx
* BridgeTowerForMaskedLM is conditioned on image too
* Add BridgeTowerForMaskedLM
* Fixes
* Call post_init to init weights
* Move freeze layers into method
* Remove BTFeatureExtractor, add BT under multimodal models
* Remove BTFeatureExtractor, add BT under multimodal models
* Code review feedback - cleanup
* Rename variables
* Formatting and style to PR review feedback
* Move center crop after resize
* Use named parameters
* Style fix for modeling_bridgetower.py
* Update docs/source/en/model_doc/bridgetower.mdx
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update docs/source/en/model_doc/bridgetower.mdx
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update docs/source/en/model_doc/bridgetower.mdx
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/bridgetower/modeling_bridgetower.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/bridgetower/modeling_bridgetower.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update docs/source/en/model_doc/bridgetower.mdx
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
* Update src/transformers/models/bridgetower/modeling_bridgetower.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Rename config params, copy BERT classes, clean comments
* Cleanup irtr
* Replace Roberta imports, add BTTextConfig and Model
* Update docs, add visionconfig, consistent arg names
* make fixup
* Comments for forward in BTModel and make fixup
* correct tests
* Remove inconsistent roberta copied from
* Add BridgeTowerTextModel to dummy_pt_objects.py
* Add BridgeTowerTextModel to IGNORE_NON_TESTED
* Update docs for BT Text and Vision Configs
* Treat BridgeTowerTextModel as a private model
* BridgeTowerTextModel as private
* Run make fix-copies
* Adding BTTextModel to PRIVATE_MODELS
* Fix for issue with BT Text and Image configs
* make style changes
* Update README_ja.md
Add から to BridgeTower's description
* Clean up config, .mdx and arg names
* Fix init_weights. Remove nn.Sequential
* Formatting and style fixes
* Re-add tie_word_embeddings in config
* update test implementation
* update style
* remove commented out
* fix style
* Update README with abs for BridgeTower
* fix style
* fix mdx file
* Update bridgetower.mdx
* Update img src in bridgetower.mdx
* Update README.md
* Update README.md
* resolve style failed
* Update _toctree.yml
* Update README_ja.md
* Removed mlp_ratio, rename feats, rename BTCLIPModel
* Replace BTCLIP with BTVisionModel,pass in vision_config to BTVisionModel
* Add test_initialization support
* Add support for output_hidden_states
* Update support for output_hidden_states
* Add support for output_attentions
* Add docstring for output_hidden_states
* update tests
* add bridgetowervisionmodel as private model
* rerun the PR test
* Remove model_type, pass configs to classes, renames
* Change self.device to use weight device
* Remove image_size
* Style check fixes
* Add hidden_size and num_hidden_layers to BridgeTowerTransformer
* Update device setting
* cosmetic update
* trigger test again
* trigger tests again
* Update test_modeling_bridgetower.py
trigger tests again
* Update test_modeling_bridgetower.py
* minor update
* re-trigger tests
* Update docs/source/en/model_doc/bridgetower.mdx
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Remove pad, update max_text_len, doc cleanup, pass eps to LayerNorm
* Added copied to, some more review feedback
* make fixup
* Use BridgeTowerVisionEmbeddings
* Code cleanup
* Fixes for BridgeTowerVisionEmbeddings
* style checks
* re-tests
* fix embedding
* address comment on init file
* retrigger tests
* update import prepare_image_inputs
* update test_image_processing_bridgetower.py to reflect test_image_processing_common.py
* retrigger tests
Co-authored-by: Shaoyen Tseng <shao-yen.tseng@intel.com>
Co-authored-by: Tiep Le <tiep.le@intel.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
Co-authored-by: Tiep Le <97980157+tileintel@users.noreply.github.com>
2023-01-25 14:04:32 -05:00
NielsRogge
f83135eb76
[Mask2Former] Add doc tests ( #21232 )
...
* Add doc tests
* Add OneFormer resourcesé
* Fix merge
* Fix style
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
2023-01-25 12:34:43 +01:00
amyeroberts
66459ce319
Add test_image_processing_common.py ( #20785 )
...
* Add test_image_processing_common.py
* Fix typo
* Update imports and test fetcher
* Revert but keep test fetcher update
* Fix imports
* Fix all imports
* Formatting fix
* Update tests/test_image_processing_common.py
2023-01-23 13:48:30 +00:00
Clémentine Fourrier
87208a05af
Graphormer model for Graph Classification ( #20968 )
...
* [FT] First commit for graphormer architecture.
The model has no tokenizer, as it uses a collator and preprocessing function for its input management.
Architecture to be tested against original one.
The arch might need to be changed to fit the checkpoint, but a revert to the original arch will make the code less nice to read.
TODO: doc
* [FIX] removed test model
* [FIX] import error
* [FIX] black and flake
* [DOC] added paper refs
* [FIX] [DOC]
* [FIX] black
* [DOC] Updated READMEs
* [FIX] Order of imports + rm Tokenizer calls
* [FIX] Moved assert in class to prevent doc build failure
* [FIX] make fix-copies
* [Doc] update from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* [FIX] Removed Graphormer from Sequence classification model list
* [DOC] Added HF copyright to Cython file
* [DOC] Fixed comments
* [FIX] typos in class doc + removed config classes.
Todo: update doc from paper definitions
* [FIX] Removed dependency to fairseq, and replaced all asserts with Exception management
* [FIX] Homogeneized initialization of weights to pretrained constructor
* [FIX] [CP] Updated multi_hop parameter to get same results as in original implementation
* [DOC] Relevant parameter description in the configuration file
* [DOC] Updated doc and comments in main graphormer file
* [FIX] make style and quality checks
* [DOC] Fix doc format
* [FIX] [WIP] Updated part of the tests, though still a wip
* [FIX] [WIP]
* [FIX] repo consistency
* [FIX] Changed input names for more understandability
* [FIX] [BUG] updated num_classes params for propagation in the model
* simplified collator
* [FIX] Updated tests to follow new naming pattern
* [TESTS] Updated test suite along with model
* |FIX] rm tokenizer import
* [DOC] add link to graphormerdoc
* Changed section in doc from text model to graph model
* Apply suggestions from code review
Spacing, inits
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* [DOC] Explain algos_graphormer functions
* Cython soft import protection
* Rm call to Callable in configuration graphormer
* [FIX] replaced asserts with Exceptions
* Add org to graphormer checkpoints
* Prefixed classes with Graphormer
* Management of init functions
* format
* fixes
* fix length file
* update indent
* relaunching ci
* Errors for missing cython imports
* fix style
* fix style doc
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-01-19 13:05:59 -05:00
Jitesh Jain
5b949623c7
Add OneFormer Model ( #20577 )
...
* Add Oneformer Model
* Add OneFormer Tests
* Add UNIVERSAL_SEGMENTATION_MAPPING
* Fix config
* 🐛 Fix error encountered while writing tests
* 🔨 Fix instance segmentation post processing
* Format Files and Add Documentation
* Add Documentation mdx file
* Run make fixup
* Run make fix-copies
* Remove unnecessary code
* Format modeling_oneformer.py
* Add OneFormer to ImageSegmentationPipeline
* Format files
* Add Demo link to Readme
* Fix fomatting errors
* Fix test failures
* Update Table in index.mdx
* Fix version
* Fix style
* Remove OneFormer from TF
* Fix Imports
* Fix dummy objects
* Fix tests
* Add newline
* Remove OneFormerFeatureExtractor
* Remove CUDA Kernels
* Use AutoBackbone for Swin
* Fix description
* Use Image Processor
* Fix copies
* Fix formatting
* Fix import order
* Fix flake8 errors
* Fix doc errors
* Add Hindi Readme entry
* Update supported backbones
* Update supported backbones
* Undo Changes
* Fix type of config
* Fix isort
* Fix auto.mdx
* Fix swin config
* Replace DinatBackbone with AutoBackbone
* Use SwinBackbone
* Use SwinBackbone
* Fix conversion script
* Fix arguments
* Add argument description
* Fix style
* Add OneFormerProcessor
* Fix OneFormerProcessor Tests
* Fix mapping
* Fix imports
* Fix inits
* Fix style
* Fix comment
* Fix docstring
* Move OneFormer to MultiModal
* Fix Copies
* Remove size divisor
* Fix check_repo.py
* Fix copies
* Add Processor for Testing Pipeline
* Fix padding for tokens
* Fix variables
* Fix formatting with correct black version
* Add Image Processor Test
* Apply suggestions
* Revert common modeling
* Add check for task
* Fix conversion script
* Fix initialization order
* Fix tests
* Undo Pipeline Changes
* Fix layers in MLP
* Fix copies
* Update image paths
* Fix copies
* Apply suggestions
2023-01-19 09:31:07 +01:00
Sylvain Gugger
05e72aa0c4
Adapt repository creation to latest hf_hub ( #21158 )
...
* Adapt repository creation to latest hf_hub
* Update all examples
* Fix other tests, add Flax examples
* Address review comments
2023-01-18 11:14:00 -05:00
NielsRogge
4ed89d48ab
Add UperNet ( #20648 )
...
* First draft
* More improvements
* Add convnext backbone
* Add conversion script
* Add more improvements
* Comment out to_dict
* Add to_dict method
* Add default config
* Fix config
* Fix backbone
* Fix backbone some more
* Add docs, auto mapping, tests
* Fix some tests
* Fix more tests
* Fix more tests
* Add conversion script
* Improve conversion script
* Add support for getting reshaped undownsampled hidden states
* Fix forward pass
* Add print statements
* Comment out set_shift_and_window_size
* More improvements
* Correct downsampling layers conversion
* Fix style
* First draft
* Fix conversion script
* Remove config attribute
* Fix more tests
* Update READMEs
* Update ConvNextBackbone
* Fix ConvNext tests
* Align ConvNext with Swin
* Remove files
* Fix index
* Improve docs
* Add output_attentions to model forward
* Add backbone mixin, improve tests
* More improvements
* Update init_weights
* Fix interpolation of logits
* Add UperNetImageProcessor
* Improve image processor
* Fix image processor
* Remove print statements
* Remove script
* Update import
* Add image processor tests
* Remove print statements
* Fix test
* Add integration test
* Add convnext integration test
* Update docstring
* Fix README
* Simplify config
* Apply suggestions
* Improve docs
* Rename class
* Fix test_initialization
* Fix import
* Address review
* Fix confg
* Convert all checkpoints
* Fix default backbone
* Usage same processor as segformer
* Apply suggestions
* Fix init_weights, update conversion scripts
* Improve config
* Use Auto API instead of creating a new image processor
* Fix docs
* Add doctests
* Remove ResNetConfig dependency
* Add always_partition argument
* Fix rebaseé
* Improve docs
* Convert checkpoints
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MBP.localdomain>
2023-01-16 09:39:13 +01:00
Yih-Dar
b3a0aad37d
Fix past CI ( #20967 )
...
* Fix for Past CI
* make style
* clean up
* unindent 2 blocks
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-01-12 18:04:21 +01:00
Yih-Dar
48d4e147d8
remove flax file from documentation_tests.txt ( #21036 )
...
remove flax file from `documentation_tests.txt`
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-01-08 12:33:25 +01:00
SABA UL HAQUE
7cb596fa22
fix typo ( #21048 )
...
Typo fix: Corrected the word metada --> metadata
2023-01-08 10:03:01 +01:00
Joao Gante
beb24f2a36
Generate: FLAX infers pad token in its absence and has functional example ( #21009 )
2023-01-05 11:52:58 +00:00
Sujay
15e17c99f9
Remove T5 dependency from mT5 model ( #20949 )
...
make mt5 independent from t5
2023-01-04 13:51:54 -05:00
Arthur
d7b66d9b44
update template ( #20885 )
...
* update template
* replace redme entries
* make style
2023-01-04 10:15:45 +01:00
Jongjyh
ce85686a1f
Add AltCLIP ( #20446 )
...
* add altclip
* update
* fix wrong title
* fix the copyright in readme
* add altclip model
* add altclip
* fix test_gradient_checkpointing_enable_disable
* code
* add return class
* add projection_state
* "fix pretrained model bug"
* delete print and fix 2 test instances.
* delete token
* rm xlmr
* one model one file.
* empty commit to trigger CI
* Fix modeling_outputs.py
* Fix __init__
* Fix quality
* Fix modeling file docstring
* Fix README.md
* Fix test file
* add vision model
* empty commit to trigger CI
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* del token in mdx file
* fix
* fix
* fix
* remove altrob from test list
* add vision test
* fix fx
* fix
* fix
* fix
* trigger CI
* fix copies
* fix tests
* fix style
* fix quality
* update
* recover import
* recover
* add ,
* recover
* fix copies
* trigger CI
* fix
* some of review
* update
* remove import
* last 2
* fix
* fix style
* fix style
* fix bug
* fix uncomment
* fix
* update
* fix
* second review
* empty commit to trigger CI
* empty commit to trigger CI
* fix position
* fix
* empty commit to trigger CI
* empty commit to trigger CI
* third comment
* Update docs/source/en/model_doc/altclip.mdx
Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>
* Update docs/source/en/model_doc/altclip.mdx
Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>
* Update src/transformers/__init__.py
Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>
* Update src/transformers/models/altclip/configuration_altclip.py
Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>
* Update src/transformers/models/altclip/modeling_altclip.py
Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>
* Update src/transformers/models/altclip/processing_altclip.py
Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>
* Update src/transformers/models/altclip/modeling_altclip.py
Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>
* fix merge
* fix copies
* update
* update
* empty commit to trigger CI
* fix code example
* empty commit to trigger CI
* fix
* empty commit to trigger CI
* empty commit to trigger CI
Co-authored-by: shunxing1234 <xw747777271@gmail.com>
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: shunxing1234 <33774367+shunxing1234@users.noreply.github.com>
Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>
2023-01-04 09:18:57 +01:00
NielsRogge
9c6f7485a6
Add GIT (GenerativeImage2Text) ( #20295 )
...
* First draft
* Make model instantiation work
* Fix copied from statement
* More fixes
* Add correct output head
* Improve configuration
* Add conversion script
* Improve conversion script
* Remove token_type_ids
* Fix conversion of projection layers
* Convert all weights
* Use cats image
* Make logits match
* Generate caption on cats image
* Add GITProcessor
* Update conversion script
* Add support for more checkpoints
* Fix conversion script
* Add initial tests
* Remove cross-attention
* More improvements
* Remove is_decoder
* Improve model tests
* Improve tests
* Improve model outputs
* Fix model outputs equivalence
* Fix more tests
* Remove unused code
* Use generate to generate text, no use of cache for now
* Use generate more appropriately
* Fix config tests
* Fix style
* Add support for use_cache
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
* Fix style
* Fix GIT vision encoder
* Update README
* Fix integration test
* Set bos and eos token ids
* Improve docs
* Improve code
* Add support for provided attention_mask
* Add copied from statement
* Fix gradient checkpointing test
* Set model_input_names
* Investigate model_input_names
* Remove script
* Fix model inputs
* Fix docstring
* Rename GIT to Git
* Support more models
* Add support for textvqa model
* Add video support
* Extend conversion script for video
* Add support for large variant
* Add support for more models
* Fix config archive map
* Update integration test
* Fix README
* Fix CLIP mean and std
* Update processor
* Fix use_cache for video, thanks @gante
* Remove print statements
* Remove assertion
* Add processor tests
* Fix model_input_names
* Use Auto API for processor
* Fix processor tests
* Fix integration test
* Fix pipeline test
* Make tests faster
* Update conversion script
* Update conversion script
* Convert more checkpoints
* Update conversion script
* Fix typo
* Update docstrings
* Improve code snippets
* Fix doc tests
* Add more code examplesé
* Fix doc tests
* Add integration tests
* Fix unused variable
* revert
* Add GIT to Japanese README
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-01-03 14:17:18 +01:00
Younes Belkada
cab7799f7b
Add japanese translation of template ( #20870 )
...
* add japanese translation of template
* fix japanese translation
- fix special cases
- fix typos
- manually translate special cases
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
2022-12-23 14:39:42 +01:00
