transformers/utils
Suraj Patil f6e74a63ca
Add m2m100 (#10236)
* m2m_100

* no layernorm_embedding

* sinusoidal positional embeddings

* update pos embeddings

* add default config values

* tokenizer

* add conversion script

* fix config

* fix pos embed

* remove _float_tensor

* update tokenizer

* update lang codes

* handle lang codes

* fix pos embeds

* fix spm key

* put embedding weights on device

* remove qa and seq classification heads

* fix convert script

* lang codes pn one line

* fix embeds

* fix tokenizer

* fix tokenizer

* add fast tokenizer

* style

* M2M100MT => M2M100

* fix copyright, style

* tokenizer converter

* vocab file

* remove fast tokenizer

* fix embeds

* fix tokenizer

* fix tests

* add tokenizer tests

* add integration test

* quality

* fix model name

* fix test

* doc

* doc

* fix doc

* add copied from statements

* fix tokenizer tests

* apply review suggestions

* fix urls

* fix shift_tokens_right

* apply review suggestions

* fix

* fix doc

* add lang code to id

* remove unused function

* update checkpoint names

* fix copy

* fix tokenizer

* fix checkpoint names

* fix merge issue

* style
2021-03-06 22:14:16 +05:30
..
tf_ops Check TF ops for ONNX compliance (#10025) 2021-02-15 07:55:10 -05:00
check_copies.py Refactor checkpoint name in BERT and MobileBERT (#10424) 2021-03-03 11:21:17 -05:00
check_dummies.py Transformers fast import part 2 (#9446) 2021-01-07 09:36:14 -05:00
check_repo.py Add m2m100 (#10236) 2021-03-06 22:14:16 +05:30
check_table.py Check table as independent script (#8976) 2020-12-07 19:55:12 -05:00
check_tf_ops.py Check TF ops for ONNX compliance (#10025) 2021-02-15 07:55:10 -05:00
class_mapping_update.py remap MODEL_FOR_QUESTION_ANSWERING_MAPPING classes to names auto-generated file (#10487) 2021-03-03 08:54:00 -08:00
download_glue_data.py GPU text generation: mMoved the encoded_prompt to correct device 2020-01-06 15:11:12 +01:00
get_modified_files.py a more reliable version of branching point discovery (#9449) 2021-01-07 04:47:50 -05:00
link_tester.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
style_doc.py Fix 9918 (#9932) 2021-02-02 05:22:20 -05:00