Tanay Mehta
7566734d6f
Add model specific output classes to PoolFormer model docs ( #15746 )
...
* Added model specific output classes to poolformer docs
* Fixed Segformer typo in Poolformer docs
2022-02-25 13:43:56 +01:00
Patrick von Platen
c44d3675c2
Time stamps for CTC models ( #15687 )
...
* [Wav2Vec2 Time Stamps]
* Add first version
* add word time stamps
* Fix
* save intermediate space
* improve
* [Finish CTC Tokenizer]
* remove @
* remove @
* push
* continue with phonemes
* up
* finish PR
* up
* add example
* rename
* finish
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* correct split
* finalize
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2022-02-22 19:26:44 +01:00
Gunjan Chhablani
ae1f835028
Add PLBart ( #13269 )
...
* Init PLBART
* Add missing configuration file
* Add conversion script and configurationf ile
* Fix style
* Update modeling and conversion scripts
* Fix scale embedding in config
* Add comment
* Fix conversion script
* Add classification option to conversion script
* Fix vocab size in config doc
* Add tokenizer files from MBart50
* Allow no lang code in regular tokenizer
* Add PLBart Tokenizer Converters
* Remove mask from multi tokenizer
* Remove mask from multi tokenizer
* Change from MBart-50 to MBart tokenizer
* Fix names and modify src/tgt behavior
* Fix imports for tokenizer
* Remove <mask> from multi tokenizer
* Fix style
* Change tokenizer_class to processor_class
* Add attribute map to config class
* Update modeling file to modified MBart code
* Update configuration file to MBart style configuration
* Fix tokenizer
* Separate tokenizers
* Fix error in tokenization auto
* Copy MBart tests
* Replace with MBart tokenization tests
* Fix style
* Fix language code in multi tokenizer
* Fix configuration docs
* Add entry for plbart_multi in transformers init
* Add dummy objects and fix imports
* Fix modeling tests
* Add TODO in config
* Fix copyright year
* Fix modeling docs and test
* Fix some tokenization tests and style
* Add changes from review
* Fix copies
* Fix docs
* Fix docs
* Fix style
* Fix year
* Add changes from review
* Remove extra changes
* Fix base tokenizer and doc
* Fix style
* Fix modeling and slow tokenizer tests
* Remove Multi-tokenizer Converter and Tests
* Delete QA model and Multi Tokenizer dummy objects
* Fix repo consistency and code quality issues
* Fix example documentation
* Fix style
* Remove PLBartTokenizer from type checking in init
* Fix consistency issue
* Add changes from review
* Fix style
* Remove PLBartTokenizerFast
* Remove FastTokenizer converter
* Fix AutoTokenzier mapping
* Add plbart to toctree and fix consistency issues
* Add language codes tokenizer test
* Fix styling and doc issues
* Add fixes for failing tests
* Fix copies
* Fix failing modeling test
* Change assert to assertTrue in modeling tests
2022-02-18 14:17:09 +01:00
NielsRogge
57882177be
Add SimMIM ( #15586 )
...
* Add first draft
* Make model importable
* Make SwinForMaskedImageModeling importable
* Fix imports
* Add missing inits
* Add support for Swin
* Fix bug
* Fix bug
* Fix another bug
* Fix Swin MIM implementation
* Fix default encoder stride
* Fix Swin
* Add print statements for debugging
* Add image_size data argument
* Fix Swin
* Fix image_size
* Add print statements for debugging
* Fix print statement
* Remove print statements
* Improve reshaping of bool_masked_pos
* Add support for DeiT, fix tests
* Improve docstrings
* Apply new black version
* Improve script
* Fix bug
* Improve README
* Apply suggestions from code review
* Remove DS_Store and add to gitignore
* Apply suggestions from code review + fix BEiT Flax
* Revert BEiT changes
* Improve README
* Fix code quality
* Improve README
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MBP.localdomain>
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
2022-02-17 19:44:55 +01:00
Tanay Mehta
f84e0dbd2a
Add PoolFormer ( #15531 )
...
* Added all files, PoolFormerFeatureExtractor still failing tests
* Fixed PoolFormerFeatureExtractor not being able to import
* Completed Poolformer doc
* Applied Suggested fixes
* Fixed errors in modeling_auto.py
* Fix feature extractor, convert docs to Markdown, styling of code
* Remove PoolFormer from check_repo and fix integration test
* Remove Poolformer from check_repo
* Fixed configuration_poolformer.py docs and removed inference.py from poolformer
* Ran with black v22
* Added PoolFormer to _toctree.yml
* Updated poolformer doc
* Applied suggested fixes and added on README.md
* Did make fixup and make fix-copies, tests should pass now
* Changed PoolFormer weights conversion script name and fixed README
* Applied fixes in test_modeling_poolformer.py and modeling_poolformer.py
* Added PoolFormerFeatureExtractor to AutoFeatureExtractor API
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MBP.localdomain>
2022-02-17 13:16:37 +01:00
Stas Bekman
bee361c6f1
[t5/t0/mt5 models] faster/leaner custom layer norm ( #14656 )
...
* [t5] faster/leaner custom layer norm
* wip
* apex.normalization.FusedRMSNorm
* cleanup
* cleanup
* add doc
* add catch all
* Trigger CI
* expand
2022-02-15 16:49:57 -08:00
jonrbates
86a7845c0c
Fix typo in speech2text2 doc ( #15617 )
...
Forward looks for inputs, not input_ids
2022-02-15 13:54:34 +01:00
NielsRogge
b090b79022
Make Swin work with VisionEncoderDecoderModel ( #15527 )
...
* Add attribute_map
* Add mention in docs
* Set hidden_size attribute correctly
* Add note about Transformer-based models only
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MBP.localdomain>
2022-02-14 17:33:35 +01:00
Steven Liu
85aee09e9a
🖍 remove broken link ( #15615 )
2022-02-11 12:33:55 -06:00
NielsRogge
a86ee2261e
Add link ( #15588 )
...
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MBP.localdomain>
2022-02-09 23:33:39 +01:00
Joao Gante
8406fa6dd5
Add TFSpeech2Text ( #15113 )
...
* Add wrapper classes
* convert inner layers to tf
* Add TF Encoder and Decoder layers
* TFSpeech2Text models
* Loadable model
* TF model with same outputs as PT model
* test skeleton
* correct tests and run the fixup
* correct attention expansion
* TFSpeech2Text pask_key_values with TF format
2022-02-08 16:27:23 +00:00
NielsRogge
84eec9e6ba
Add ConvNeXT ( #15277 )
...
* First draft
* Add conversion script
* Improve conversion script
* Improve docs and implement tests
* Define model output class
* Fix tests
* Fix more tests
* Add model to README
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Apply more suggestions from code review
* Apply suggestions from code review
* Rename dims to hidden_sizes
* Fix equivalence test
* Rename gamma to gamma_parameter
* Clean up conversion script
* Add ConvNextFeatureExtractor
* Add corresponding tests
* Implement feature extractor correctly
* Make implementation cleaner
* Add ConvNextStem class
* Improve design
* Update design to also include encoder
* Fix gamma parameter
* Use sample docstrings
* Finish conversion, add center cropping
* Replace nielsr by facebook, make feature extractor tests smaller
* Fix integration test
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2022-02-07 16:11:37 +01:00
Sylvain Gugger
ac6aa10f23
Standardize semantic segmentation models outputs ( #15469 )
...
* Standardize instance segmentation models outputs
* Rename output
* Update src/transformers/modeling_outputs.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Add legacy argument to the config and model forward
* Update src/transformers/models/beit/modeling_beit.py
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
* Copy fix in Segformer
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
2022-02-04 14:52:07 -05:00
NielsRogge
47df0f2234
Add header ( #15434 )
2022-01-31 11:15:54 -05:00
Soonhwan-Kwon
e09473a817
Add support for XLM-R XL and XXL models by modeling_xlm_roberta_xl.py ( #13727 )
...
* add xlm roberta xl
* add convert xlm xl fairseq checkpoint to pytorch
* fix init and documents for xlm-roberta-xl
* fix indention
* add test for XLM-R xl,xxl
* fix model hub name
* fix some stuff
* up
* correct init
* fix more
* fix as suggestions
* add torch_device
* fix default values of doc strings
* fix leftovers
* merge to master
* up
* correct hub names
* fix docs
* fix model
* up
* finalize
* last fix
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* add copied from
* make style
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2022-01-29 13:42:37 +01:00
Suraj Patil
d25e25ee2b
Add XGLM models ( #14876 )
...
* add xglm
* update vocab size
* fix model name
* style and tokenizer
* typo
* no mask token
* fix pos embed compute
* fix args
* fix tokenizer
* fix positions
* fix tokenization
* style and dic fixes
* fix imports
* add fast tokenizer
* update names
* add pt tests
* fix tokenizer
* fix typo
* fix tokenizer import
* fix fast tokenizer
* fix tokenizer
* fix converter
* add tokenizer test
* update checkpoint names
* fix tokenizer tests
* fix slow tests
* add copied from comments
* rst -> mdx
* flax model
* update flax tests
* quality
* style
* doc
* update index and readme
* fix copies
* fix doc
* update toctrr
* fix indent
* minor fixes
* fix config doc
* don't save embed_pos weights
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* address Sylvains commnets, few doc fixes
* fix check_repo
* align order of arguments
* fix copies
* fix labels
* remove unnecessary mapping
* fix saving tokenizer
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2022-01-28 18:55:23 +01:00
novice
99a2771189
Add YOSO ( #15091 )
...
* Add cookiecutter files
* Add cuda kernels and cpp files
* Update modeling_yoso.py
* Add .h files
* Update configuration_yoso.py
* Updates
* Remove tokenizer
* Code quality
* Update modeling_yoso.py
* Update modeling_yoso.py
* Fix failing test
* Update modeling_yoso.py
* Fix code quality
* Apply suggestions from code review
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Apply suggestions from code review
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Apply suggestions from code review and fix integration tests
* Update src/transformers/models/yoso/modeling_yoso.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* Apply suggestions from code review
* Fix copied from statement
* Fix docstring
* Fix code quality
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Apply suggestions and fix mask
* Apply suggestions from code review
* Fix code quality
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Fix docstrings
* Fix code quality
* Remove trailing whitespace
* Update yoso.mdx
* Move kernel loading to YosoEncoder
* make style
* Apply suggestions from code review
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/yoso/modeling_yoso.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Add short summary to docs
* Update docs/source/model_doc/yoso.mdx
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update yoso.mdx
* Update docs/source/model_doc/yoso.mdx
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Remove CausalLM model and add copied from
* Remove autoregressive code
* Remove unused imports
* add copied from for embeddings
* Fix code quality
* Update docs/source/model_doc/yoso.mdx
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Apply suggestion from code review
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2022-01-26 19:18:29 +01:00
Jacob Deppen
96161ac408
make table into valid Markdown table syntax ( #15337 )
2022-01-26 07:10:00 -05:00
novice
d43e308e7f
Add Swin Transformer ( #15085 )
...
* Add all files
* Apply suggestions from code review
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Updates
* Apply suggestions from review
* Fix failing tests
* Update __init__.py
* Update configuration_swin.py
* Update auto_factory.py
* Fix pytests
* Apply suggestions from code review
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Fix tests and default checkpoint
* Fix Recursion error
* Code quality
* Remove copied from
* Update modeling_swin.py
* Code quality
* Update modeling_swin.py
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Apply suggestions from code review
* Fix feature extractor
* Fix code quality
* Apply suggestions from code review
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Apply suggestions from code review
* Update configuration_swin.py
* Update default checkpoint
* Apply suggestions from code review
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update docs/source/model_doc/swin.mdx
Co-authored-by: Mishig Davaadorj <mishig.davaadorj@coloradocollege.edu>
* Update conversion script
* Reformat conversion script
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Mishig Davaadorj <mishig.davaadorj@coloradocollege.edu>
2022-01-21 12:10:41 +01:00
NielsRogge
515ed3ad2a
Fix doc examples ( #15257 )
2022-01-20 21:51:51 +01:00
NielsRogge
ac227093e4
Add ViLT ( #14895 )
...
* First commit
* Add conversion script
* Make conversion script work for base model
* More improvements
* Update conversion script, works for vqa
* Add indexing argument to meshgrid
* Make conversion script work for ViltForPreTraining
* Add ViltForPreTraining to docs
* Fix device issue
* Add processor
* Add MinMaxResize to feature extractor
* Implement call method of ViltProcessor
* Fix tests
* Add integration test
* Add loss calculation for VQA
* Improve tests
* Improve some more tests
* Debug tests
* Small improvements
* Add support for attention_mask
* Remove mask_it
* Add pixel_mask
* Add tests for ViltFeatureExtractor
* Improve tests
* Add ViltForNaturalLanguageVisualReasoning
* Add ViltForNaturalLanguageVisualReasoning to conversion script
* Minor fixes
* Add support for image_embeds, update docstrings to markdown
* Update docs to markdown
* Improve conversion script
* Rename ViltForPreTraining to ViltForMaskedLM
* Improve conversion script
* Convert docstrings to markdown
* Fix code example of retrieval model
* Properly convert masked language model
* Add integration test for nlvr
* Fix code quality
* Apply suggestions from code review
* Add copied from statements
* Fix pretrained_config_archive_map
* Fix docs
* Add model to README
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Apply more suggestions from code review
* Make code more readable
* Add ViltForNaturalLanguageVisualReasoning to the tests
* Rename ViltForVisualQuestionAnswering to ViltForQuestionAnswering
* Replace pixel_values_2 by single tensor
* Add hidden_states and attentions
* Fix one more test
* Fix all tests
* Update year
* Fix rebase issues
* Fix another rebase issue
* Remove ViltForPreTraining from auto mapping
* Rename ViltForImageRetrievalTextRetrieval to ViltForImageAndTextRetrieval
* Make it possible to use BertTokenizerFast in the processor
* Use BertTokenizerFast by default
* Rename ViltForNaturalLanguageVisualReasoning, define custom model output
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2022-01-19 19:51:59 +01:00
NielsRogge
842298f84f
[ViTMAE] Various fixes ( #15221 )
...
* Add MAE to AutoFeatureExtractor
* Add link to notebook
* Fix relative paths
2022-01-19 15:27:57 +01:00
Li-Huai (Allan) Lin
841d979190
Add FastTokenizer to REALM ( #15211 )
...
* Remove BertTokenizer abstraction
* Add FastTokenizer to REALM
* Fix config archive map
* Fix copies
* Update realm.mdx
* Apply suggestions from code review
2022-01-19 15:19:36 +01:00
Sylvain Gugger
db3503949d
Finish conversion of REALM doc to MDX
2022-01-18 18:00:30 -05:00
NielsRogge
74bec9865c
Add MAE ( #15120 )
...
* First draft
* More improvements
* More improvements
* More improvements
* Fix embeddings
* Add conversion script
* Finish conversion script
* More improvements
* Fix forward pass
* Remove print statements
* Add weights initialization
* Add initialization of decoder weights
* Add support for other models in the conversion script
* Fix patch_size for huge model
* Fix most of the tests
* Fix integration test
* Fix docs
* Fix archive_list
* Apply suggestions from code review
* Improve documentation
* Apply more suggestions
* Skip some tests due to non-deterministic behaviour
* Fix test_initialization
* Remove unneccessary initialization of nn.Embedding
* Improve docs
* Fix dummies
* Remove ViTMAEFeatureExtractor from docs
* Add model to README and table of contents
* Delete inference file
2022-01-18 16:21:32 +01:00
Li-Huai (Allan) Lin
22454ae492
Add REALM ( #13292 )
...
* REALM initial commit
* Retriever OK (Update new_gelu).
* Encoder prediction score OK
* Encoder pretrained model OK
* Update retriever comments
* Update docs, tests, and imports
* Prune unused models
* Make embedder as a module `RealmEmbedder`
* Add RealmRetrieverOutput
* Update tokenization
* Pass all tests in test_modeling_realm.py
* Prune RealmModel
* Update docs
* Add training test.
* Remove completed TODO
* Style & Quality
* Prune `RealmModel`
* Fixup
* Changes:
1. Remove RealmTokenizerFast
2. Update docstrings
3. Add a method to RealmTokenizer to handle candidates tokenization.
* Fix up
* Style
* Add tokenization tests
* Update `from_pretrained` tests
* Apply suggestions
* Style & Quality
* Copy BERT model
* Fix comment to avoid docstring copying
* Make RealmBertModel private
* Fix bug
* Style
* Basic QA
* Save
* Complete reader logits
* Add searcher
* Complete searcher & reader
* Move block records init to constructor
* Fix training bug
* Add some outputs to RealmReader
* Add finetuned checkpoint variable names parsing
* Fix bug
* Update REALM config
* Add RealmForOpenQA
* Update convert_tfrecord logits
* Fix bugs
* Complete imports
* Update docs
* Update naming
* Add brute-force searcher
* Pass realm model tests
* Style
* Exclude RealmReader from common tests
* Fix
* Fix
* convert docs
* up
* up
* more make style
* up
* upload
* up
* Fix
* Update src/transformers/__init__.py
* adapt testing
* change modeling code
* fix test
* up
* up
* up
* correct more
* make retriever work
* update
* make style
* finish main structure
* Resolve merge conflict
* Make everything work
* Style
* Fixup
* Fixup
* Update training test
* fix retriever
* remove hardcoded path
* Fix
* Fix modeling test
* Update model links
* Initial retrieval test
* Fix modeling test
* Complete retrieval tests
* Fix
* style
* Fix tests
* Fix docstring example
* Minor fix of retrieval test
* Update license headers and docs
* Apply suggestions from code review
* Style
* Apply suggestions from code review
* Add an example to RealmEmbedder
* Fix
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2022-01-18 07:24:13 -05:00
novice
28e091430e
Add Nystromformer ( #14659 )
...
* Initial commit
* Config and modelling changes
Added Nystromformer-specific attributes to config and removed all decoder functionality from modelling.
* Modelling and test changes
Added Nystrom approximation and removed decoder tests.
* Code quality fixes
* Modeling changes and conversion script
Initial commits to conversion script, modeling changes.
* Minor modeling changes and conversion script
* Modeling changes
* Correct modeling, add tests and documentation
* Code refactor
* Remove tokenizers
* Code refactor
* Update __init__.py
* Fix bugs
* Update src/transformers/__init__.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/__init__.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/nystromformer/__init__.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update docs/source/model_doc/nystromformer.mdx
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/nystromformer/configuration_nystromformer.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/nystromformer/configuration_nystromformer.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/nystromformer/configuration_nystromformer.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/nystromformer/configuration_nystromformer.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/nystromformer/convert_nystromformer_original_pytorch_checkpoint_to_pytorch.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/nystromformer/configuration_nystromformer.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update modeling and test_modeling
* Code refactor
* .rst to .mdx
* doc changes
* Doc changes
* Update modeling_nystromformer.py
* Doc changes
* Fix copies
* Apply suggestions from code review
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update configuration_nystromformer.py
* Fix copies
* Update tests/test_modeling_nystromformer.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update test_modeling_nystromformer.py
* Apply suggestions from code review
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
* Fix code style
* Update modeling_nystromformer.py
* Update modeling_nystromformer.py
* Fix code style
* Reformat modeling file
* Update modeling_nystromformer.py
* Modify NystromformerForMultipleChoice
* Fix code quality
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Code style changes and torch.no_grad()
* make style
* Apply suggestions from code review
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2022-01-11 14:25:49 +01:00
Yih-Dar
b67fd797be
Add TFVisionEncoderDecoderModel ( #14148 )
...
* Start the work on TFVisionEncoderDecoderModel
* Expose TFVisionEncoderDecoderModel
* fix import
* Add modeling_tf_vision_encoder_decoder to _ignore_modules in get_model_modules()
* reorder
* Apply the fix for checkpoint loading as in #14016
* remove attention_mask + fix VISION_DUMMY_INPUTS
* A minimal change to make TF generate() work for vision models as encoder in encoder-decoder setting
* fix wrong condition: shape_list(input_ids) == 2
* add tests
* use personal TFViTModel checkpoint (for now)
* Add equivalence tests + projection layer
* style
* make sure projection layer can run
* Add examples
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Clean comments (need to work on TODOs for PyTorch models)
* Remove TF -> PT in check_pt_tf_equivalence for TFVisionEncoderDecoderModel
* fixes
* Revert changes in PT code.
* Update tests/test_modeling_tf_vision_encoder_decoder.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* Add test_inference_coco_en for TF test
* fix quality
* fix name
* build doc
* add main_input_name
* Fix ckpt name in test
* fix diff between master and this PR
* fix doc
* fix style and quality
* fix missing doc
* fix labels handling
* Delete auto.rst
* Add the changes done in #14016
* fix prefix
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* make style
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2022-01-10 13:30:14 -05:00
Minghao Li
b2c477fc6d
support the trocr small models ( #14893 )
...
* support the trocr small models
* resolve conflict
* Update docs/source/model_doc/trocr.mdx
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update docs/source/model_doc/trocr.mdx
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update docs/source/model_doc/trocr.mdx
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/trocr/processing_trocr.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/trocr/processing_trocr.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/trocr/processing_trocr.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/trocr/processing_trocr.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* fix unexpected indent in processing_trocr.py
* Update src/transformers/models/trocr/processing_trocr.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* update the docstring of processing_trocr
* remove extra space
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
2022-01-10 09:28:03 -05:00
Daniel Stancl
21aecc0971
Add Flax RoFormer ( #15005 )
...
* Add FlaxRoFormer
* Clean code + make quality
* Fix output pooling for FlaxRoFormerForMultipleChoiceModule
* Apply suggestions from code review
* add flax model to repos
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2022-01-04 13:23:10 +01:00
Sylvain Gugger
8f6373c61c
Map model_type and doc pages names ( #14944 )
...
* Map model_type and doc pages names
* Add script
* Fix typo
* Quality
* Manual check for Auto
Co-authored-by: Lysandre <lysandre.debut@reseau.eseo.fr>
2022-01-03 05:08:55 -05:00
Sylvain Gugger
2c5597f6c7
Style
2021-12-27 19:18:08 -05:00
Sylvain Gugger
b5e2b183af
Doc styler examples ( #14953 )
...
* Fix bad examples
* Add black formatting to style_doc
* Use first nonempty line
* Put it at the right place
* Don't add spaces to empty lines
* Better templates
* Deal with triple quotes in docstrings
* Result of style_doc
* Enable mdx treatment and fix code examples in MDXs
* Result of doc styler on doc source files
* Last fixes
* Break copy from
2021-12-27 19:07:46 -05:00
Stas Bekman
133c5e40c4
[doc] consistent True/False/None default format ( #14951 )
...
* [doc] consistent True/False/None default format
* Update src/transformers/models/xlnet/modeling_xlnet.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2021-12-27 14:31:40 -08:00
Sylvain Gugger
b2f500256e
Convert last rst file ( #14952 )
2021-12-27 17:09:37 -05:00
Daniel Stancl
501307b58b
Add ElectraForCausalLM -> Enable Electra encoder-decoder model ( #14729 )
...
* Add ElectraForCausalLM and cover some basic tests & need to fix a few tests
* Fix bugs
* make style
* make fix-copies
* Update doc
* Change docstring to markdown format
* Remove redundant update_keys_to_ignore
2021-12-27 12:37:52 +01:00
Yih-Dar
8f2cc1c3ab
Add TFCLIPModel ( #13967 )
...
* Start the work for TFCLIPModel
* Convert to TF code (TODO: loss + doc)
* Clean up
* Fix pooled_output for TFCLIPTextTransformer - using tf.gather_nd
* assert -> raise error
* Expose TFCLIPModel
* Deal with dummy_inputs
* Add tests
* Fix all tests. TODO: manual check weight loading + add more comments
* Fix pt tf equivalence test
* fixes
* update TFCLIPVisionEmbeddings's Conv2D
* Fix loss + overwrite test_pt_tf_model_equivalence from common
* Add a comment about the change about MainLayer in test_keras_save_load
* Set return_loss=True in TFCLIPModelTester + make tests pass
* overwrite test_pt_tf_model_equivalence from tf common
* fix base_model_prefix
* Fix examples
* remove unused
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* apply review suggestions
* change self.pre_layrnorm to self.pre_layernorm
* apply more review suggestions
* return attention probs before dropout (to align with PT)
* fix weight init
* fix
* build doc
* fix missing doc
* fix for test
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2021-12-23 11:19:44 -05:00
Sylvain Gugger
207594be81
Convert rst files ( #14888 )
...
* Convert all tutorials and guides
* Convert all remaining rst to mdx
* Track and fix bad links
2021-12-22 16:14:35 -05:00
NielsRogge
7df4b90c76
Fix Perceiver docs ( #14879 )
2021-12-22 14:18:03 +01:00
Lysandre Debut
ec3567fe20
Convert model files from rst to mdx ( #14865 )
...
* First pass
* Apply suggestions from code review
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2021-12-22 03:27:30 -05:00
Patrick von Platen
952a77b05d
[Perceiver] Skip multi-gpu tests for now ( #14813 )
...
* [Perceiver] Skip multi-gpu tests for now
* Update tests/test_modeling_perceiver.py
* up
* up
2021-12-20 15:22:50 +01:00
Anton Lozhkov
3883e3a75e
Add SD and SV heads for WavLM ( #14847 )
...
* Add converted heads
* Add dummies
2021-12-20 16:40:56 +03:00
Patrick von Platen
c4a96cecbc
Wav2Vec2 meets phonemes ( #14353 )
...
* up
* add tokenizer
* improve more
* finish tokenizer
* finish
* adapt speech recognition script
* adapt convert
* more fixes
* more fixes
* update phonemizer wav2vec2
* better naming
* fix more tests
* more fixes swedish
* correct tests
* finish
* improve script
* remove file
* up
* lets get those 100 model architectures until the end of the month
* make fix-copies
* correct more
* correct script
* more fixes
* more fixes
* add to docs
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* replace assert
* fix copies
* fix docs
* new try docs
* boom boom
* update
* add phonemizer to audio tests
* make fix-copies
* up
* upload models
* some changes
* Update tests/test_tokenization_wav2vec2_phoneme.py
Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
* more fixes
* remove @
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
2021-12-17 19:56:44 +01:00
Lysandre Debut
77d6c826d8
Convert rst to mdx bert ( #14806 )
...
* BERT to mdx
mdx :)
c
* Update docs/source/model_doc/bert.mdx
Co-authored-by: Julien Chaumond <julien@huggingface.co>
* Remove all
Co-authored-by: sgugger <sylvain.gugger@gmail.com>
Co-authored-by: Julien Chaumond <julien@huggingface.co>
2021-12-17 11:13:34 -05:00
Patrick von Platen
bef1e3e4a0
Add WavLM ( #14354 )
...
* first commit
* fix some stuff
* fix more readme
* Apply suggestions from code review
* update
* correct
* up
* attn layer works
* push code
* make modedls work
* Small change
* more refactor
* finish
* up
* fix convertsion
* fix position bias
* Fix style
* fix conversion
* make fix-copies
* add
* clean
* fix docs
* fix
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* apply final changes
* make fix-copies
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2021-12-16 18:57:05 +01:00
Anton Lozhkov
48463ebb33
Add Speaker Diarization and Verification heads ( #14723 )
...
* Models
* Squashed commit of the following:
commit 72278e1e931a16d0879acc77f65762f3364833d0
Author: anton-l <aglozhkov@gmail.com>
Date: Fri Dec 10 21:45:08 2021 +0300
* Add unispeech heads
* Add sd/sv automodels
* Docs cleanup
* Fix docstrings
* rename xvector classes
* examples
* Tests cleanup
* Style
* Better checkpoints for tests
* leftover docs
* apply review suggestions
* Style + init tests
* Update unispeech-sat tdnn downsampling
2021-12-16 19:22:14 +03:00
NielsRogge
50bc57cef8
Update Perceiver code examples ( #14783 )
...
* Fix code examples
* Fix code example
2021-12-15 11:06:38 -05:00
Xing Han Lu
72c6e8b8bf
Update t5.rst ( #14776 )
2021-12-15 14:59:11 +01:00
Lysandre Debut
6e05bb1c96
Fix the perceiver docs ( #14748 )
2021-12-13 09:29:47 -05:00
NielsRogge
4c99e553c1
Improve documentation of some models ( #14695 )
...
* Migrate docs to mdx
* Update TAPAS docs
* Remove lines
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Apply some more suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Add pt/tf switch to code examples
* More improvements
* Improve docstrings
* More improvements
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2021-12-13 13:24:36 +01:00
