novice
99a2771189
Add YOSO ( #15091 )
...
* Add cookiecutter files
* Add cuda kernels and cpp files
* Update modeling_yoso.py
* Add .h files
* Update configuration_yoso.py
* Updates
* Remove tokenizer
* Code quality
* Update modeling_yoso.py
* Update modeling_yoso.py
* Fix failing test
* Update modeling_yoso.py
* Fix code quality
* Apply suggestions from code review
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Apply suggestions from code review
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Apply suggestions from code review and fix integration tests
* Update src/transformers/models/yoso/modeling_yoso.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* Apply suggestions from code review
* Fix copied from statement
* Fix docstring
* Fix code quality
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Apply suggestions and fix mask
* Apply suggestions from code review
* Fix code quality
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Fix docstrings
* Fix code quality
* Remove trailing whitespace
* Update yoso.mdx
* Move kernel loading to YosoEncoder
* make style
* Apply suggestions from code review
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/yoso/modeling_yoso.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Add short summary to docs
* Update docs/source/model_doc/yoso.mdx
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update yoso.mdx
* Update docs/source/model_doc/yoso.mdx
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Remove CausalLM model and add copied from
* Remove autoregressive code
* Remove unused imports
* add copied from for embeddings
* Fix code quality
* Update docs/source/model_doc/yoso.mdx
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Apply suggestion from code review
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2022-01-26 19:18:29 +01:00
Steven Liu
0501beb846
Add 🤗 Accelerate tutorial ( #15263 )
...
* add accelerate tutorial
* 🖍 apply feedback from review
* 📝 make edits
2022-01-25 13:46:11 -06:00
novice
d43e308e7f
Add Swin Transformer ( #15085 )
...
* Add all files
* Apply suggestions from code review
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Updates
* Apply suggestions from review
* Fix failing tests
* Update __init__.py
* Update configuration_swin.py
* Update auto_factory.py
* Fix pytests
* Apply suggestions from code review
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Fix tests and default checkpoint
* Fix Recursion error
* Code quality
* Remove copied from
* Update modeling_swin.py
* Code quality
* Update modeling_swin.py
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Apply suggestions from code review
* Fix feature extractor
* Fix code quality
* Apply suggestions from code review
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Apply suggestions from code review
* Update configuration_swin.py
* Update default checkpoint
* Apply suggestions from code review
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update docs/source/model_doc/swin.mdx
Co-authored-by: Mishig Davaadorj <mishig.davaadorj@coloradocollege.edu>
* Update conversion script
* Reformat conversion script
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Mishig Davaadorj <mishig.davaadorj@coloradocollege.edu>
2022-01-21 12:10:41 +01:00
NielsRogge
ac227093e4
Add ViLT ( #14895 )
...
* First commit
* Add conversion script
* Make conversion script work for base model
* More improvements
* Update conversion script, works for vqa
* Add indexing argument to meshgrid
* Make conversion script work for ViltForPreTraining
* Add ViltForPreTraining to docs
* Fix device issue
* Add processor
* Add MinMaxResize to feature extractor
* Implement call method of ViltProcessor
* Fix tests
* Add integration test
* Add loss calculation for VQA
* Improve tests
* Improve some more tests
* Debug tests
* Small improvements
* Add support for attention_mask
* Remove mask_it
* Add pixel_mask
* Add tests for ViltFeatureExtractor
* Improve tests
* Add ViltForNaturalLanguageVisualReasoning
* Add ViltForNaturalLanguageVisualReasoning to conversion script
* Minor fixes
* Add support for image_embeds, update docstrings to markdown
* Update docs to markdown
* Improve conversion script
* Rename ViltForPreTraining to ViltForMaskedLM
* Improve conversion script
* Convert docstrings to markdown
* Fix code example of retrieval model
* Properly convert masked language model
* Add integration test for nlvr
* Fix code quality
* Apply suggestions from code review
* Add copied from statements
* Fix pretrained_config_archive_map
* Fix docs
* Add model to README
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Apply more suggestions from code review
* Make code more readable
* Add ViltForNaturalLanguageVisualReasoning to the tests
* Rename ViltForVisualQuestionAnswering to ViltForQuestionAnswering
* Replace pixel_values_2 by single tensor
* Add hidden_states and attentions
* Fix one more test
* Fix all tests
* Update year
* Fix rebase issues
* Fix another rebase issue
* Remove ViltForPreTraining from auto mapping
* Rename ViltForImageRetrievalTextRetrieval to ViltForImageAndTextRetrieval
* Make it possible to use BertTokenizerFast in the processor
* Use BertTokenizerFast by default
* Rename ViltForNaturalLanguageVisualReasoning, define custom model output
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2022-01-19 19:51:59 +01:00
NielsRogge
74bec9865c
Add MAE ( #15120 )
...
* First draft
* More improvements
* More improvements
* More improvements
* Fix embeddings
* Add conversion script
* Finish conversion script
* More improvements
* Fix forward pass
* Remove print statements
* Add weights initialization
* Add initialization of decoder weights
* Add support for other models in the conversion script
* Fix patch_size for huge model
* Fix most of the tests
* Fix integration test
* Fix docs
* Fix archive_list
* Apply suggestions from code review
* Improve documentation
* Apply more suggestions
* Skip some tests due to non-deterministic behaviour
* Fix test_initialization
* Remove unneccessary initialization of nn.Embedding
* Improve docs
* Fix dummies
* Remove ViTMAEFeatureExtractor from docs
* Add model to README and table of contents
* Delete inference file
2022-01-18 16:21:32 +01:00
Li-Huai (Allan) Lin
22454ae492
Add REALM ( #13292 )
...
* REALM initial commit
* Retriever OK (Update new_gelu).
* Encoder prediction score OK
* Encoder pretrained model OK
* Update retriever comments
* Update docs, tests, and imports
* Prune unused models
* Make embedder as a module `RealmEmbedder`
* Add RealmRetrieverOutput
* Update tokenization
* Pass all tests in test_modeling_realm.py
* Prune RealmModel
* Update docs
* Add training test.
* Remove completed TODO
* Style & Quality
* Prune `RealmModel`
* Fixup
* Changes:
1. Remove RealmTokenizerFast
2. Update docstrings
3. Add a method to RealmTokenizer to handle candidates tokenization.
* Fix up
* Style
* Add tokenization tests
* Update `from_pretrained` tests
* Apply suggestions
* Style & Quality
* Copy BERT model
* Fix comment to avoid docstring copying
* Make RealmBertModel private
* Fix bug
* Style
* Basic QA
* Save
* Complete reader logits
* Add searcher
* Complete searcher & reader
* Move block records init to constructor
* Fix training bug
* Add some outputs to RealmReader
* Add finetuned checkpoint variable names parsing
* Fix bug
* Update REALM config
* Add RealmForOpenQA
* Update convert_tfrecord logits
* Fix bugs
* Complete imports
* Update docs
* Update naming
* Add brute-force searcher
* Pass realm model tests
* Style
* Exclude RealmReader from common tests
* Fix
* Fix
* convert docs
* up
* up
* more make style
* up
* upload
* up
* Fix
* Update src/transformers/__init__.py
* adapt testing
* change modeling code
* fix test
* up
* up
* up
* correct more
* make retriever work
* update
* make style
* finish main structure
* Resolve merge conflict
* Make everything work
* Style
* Fixup
* Fixup
* Update training test
* fix retriever
* remove hardcoded path
* Fix
* Fix modeling test
* Update model links
* Initial retrieval test
* Fix modeling test
* Complete retrieval tests
* Fix
* style
* Fix tests
* Fix docstring example
* Minor fix of retrieval test
* Update license headers and docs
* Apply suggestions from code review
* Style
* Apply suggestions from code review
* Add an example to RealmEmbedder
* Fix
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2022-01-18 07:24:13 -05:00
lewtun
021f2ea987
Add ONNX configuration classes to docs ( #15121 )
...
* Add ONNX classes to main package
* Remove permalinks from ONNX guide
* Fix ToC entry
* Revert "Add ONNX classes to main package"
This reverts commit eb794a5b00
2022-01-12 16:33:32 +01:00
novice
28e091430e
Add Nystromformer ( #14659 )
...
* Initial commit
* Config and modelling changes
Added Nystromformer-specific attributes to config and removed all decoder functionality from modelling.
* Modelling and test changes
Added Nystrom approximation and removed decoder tests.
* Code quality fixes
* Modeling changes and conversion script
Initial commits to conversion script, modeling changes.
* Minor modeling changes and conversion script
* Modeling changes
* Correct modeling, add tests and documentation
* Code refactor
* Remove tokenizers
* Code refactor
* Update __init__.py
* Fix bugs
* Update src/transformers/__init__.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/__init__.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/nystromformer/__init__.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update docs/source/model_doc/nystromformer.mdx
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/nystromformer/configuration_nystromformer.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/nystromformer/configuration_nystromformer.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/nystromformer/configuration_nystromformer.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/nystromformer/configuration_nystromformer.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/nystromformer/convert_nystromformer_original_pytorch_checkpoint_to_pytorch.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/nystromformer/configuration_nystromformer.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update modeling and test_modeling
* Code refactor
* .rst to .mdx
* doc changes
* Doc changes
* Update modeling_nystromformer.py
* Doc changes
* Fix copies
* Apply suggestions from code review
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update configuration_nystromformer.py
* Fix copies
* Update tests/test_modeling_nystromformer.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update test_modeling_nystromformer.py
* Apply suggestions from code review
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
* Fix code style
* Update modeling_nystromformer.py
* Update modeling_nystromformer.py
* Fix code style
* Reformat modeling file
* Update modeling_nystromformer.py
* Modify NystromformerForMultipleChoice
* Fix code quality
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Code style changes and torch.no_grad()
* make style
* Apply suggestions from code review
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2022-01-11 14:25:49 +01:00
Sylvain Gugger
8f6373c61c
Map model_type and doc pages names ( #14944 )
...
* Map model_type and doc pages names
* Add script
* Fix typo
* Quality
* Manual check for Auto
Co-authored-by: Lysandre <lysandre.debut@reseau.eseo.fr>
2022-01-03 05:08:55 -05:00
Ryokan RI
824fd44fc3
Feature/fix slow test in mluke ( #14749 )
...
* make MLukeTokenizerTest fast
* make LukeTokenizerTest fast
* add entry to _toctree.yaml
2021-12-22 06:35:59 -05:00
Patrick von Platen
c4a96cecbc
Wav2Vec2 meets phonemes ( #14353 )
...
* up
* add tokenizer
* improve more
* finish tokenizer
* finish
* adapt speech recognition script
* adapt convert
* more fixes
* more fixes
* update phonemizer wav2vec2
* better naming
* fix more tests
* more fixes swedish
* correct tests
* finish
* improve script
* remove file
* up
* lets get those 100 model architectures until the end of the month
* make fix-copies
* correct more
* correct script
* more fixes
* more fixes
* add to docs
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* replace assert
* fix copies
* fix docs
* new try docs
* boom boom
* update
* add phonemizer to audio tests
* make fix-copies
* up
* upload models
* some changes
* Update tests/test_tokenization_wav2vec2_phoneme.py
Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
* more fixes
* remove @
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
2021-12-17 19:56:44 +01:00
Patrick von Platen
bef1e3e4a0
Add WavLM ( #14354 )
...
* first commit
* fix some stuff
* fix more readme
* Apply suggestions from code review
* update
* correct
* up
* attn layer works
* push code
* make modedls work
* Small change
* more refactor
* finish
* up
* fix convertsion
* fix position bias
* Fix style
* fix conversion
* make fix-copies
* add
* clean
* fix docs
* fix
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* apply final changes
* make fix-copies
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2021-12-16 18:57:05 +01:00
Sylvain Gugger
322d416916
Update Table of Contents ( #14755 )
2021-12-13 17:15:19 -05:00
Mishig Davaadorj
60be4bf8ac
Fix typo in toctree ( #14704 )
2021-12-09 09:25:31 -05:00
Sylvain Gugger
01b8cd5932
Revert open-in-colab and add perceiver ( #14683 )
2021-12-08 13:52:31 -05:00
Mishig Davaadorj
4f68de625c
Rename toctree.yml -> _toctree.yml ( #14594 )
2021-12-02 08:58:39 +01:00
