yujun
|
206f06f2dd
|
Add new model RoFormer (use rotary position embedding ) (#11684)
* add roformer
* Update docs/source/model_doc/roformer.rst
Co-authored-by: Suraj Patil <surajp815@gmail.com>
* Update docs/source/model_doc/roformer.rst
Co-authored-by: Suraj Patil <surajp815@gmail.com>
* update
* add TFRoFormerSinusoidalPositionalEmbedding and fix TFMarianSinusoidalPositionalEmbedding
* update docs
* make style and make quality
* roback
* unchanged
* rm copies from , this is a error in TFMarianSinusoidalPositionalEmbedding
* update Copyright year
* move # Add modeling imports here to the correct position
* max_position_embeddings can be set to 1536
* # Copied from transformers.models.bert.modeling_bert.BertOutput with Bert->RoFormer
* # Copied from transformers.models.bert.modeling_bert.BertLayer.__init__ with Bert->RoFormer
* update tokenization_roformer
* make style
* add staticmethod apply_rotary_position_embeddings
* add TF staticmethod apply_rotary_position_embeddings
* update torch apply_rotary_position_embeddings
* fix tf apply_rotary_position_embeddings error
* make style
* add pytorch RoFormerSelfAttentionRotaryPositionEmbeddingTest
* add TF rotary_position_embeddings test
* update test_modeling_rofomer
* Update docs/source/model_doc/roformer.rst
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/__init__.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/__init__.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/__init__.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/__init__.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/models/roformer/convert_roformer_original_tf_checkpoint_to_pytorch.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/models/roformer/modeling_roformer.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/models/roformer/modeling_roformer.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/models/roformer/modeling_tf_roformer.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* refact roformer tokenizer
* add RoFormerTokenizerFast
* add RoFormerTokenizationTest
* add require_jieba
* update Copyright
* update tokenizer & add copy from
* add option rotary_value
* use rust jieba
* use rjieba
* use rust jieba
* fix test_alignement_methods
* slice normalized_string is too slow
* add config.embedding_size when embedding_size!=hidden_size
* fix pickle tokenizer
* Update docs/source/model_doc/roformer.rst
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* make style and make quality
Co-authored-by: Suraj Patil <surajp815@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
|
2021-05-20 08:00:34 -04:00 |
|