transformers/utils
Gunjan Chhablani ae1f835028
Add PLBart (#13269)
* Init PLBART

* Add missing configuration file

* Add conversion script and configurationf ile

* Fix style

* Update modeling and conversion scripts

* Fix scale embedding in config

* Add comment

* Fix conversion script

* Add classification option to conversion script

* Fix vocab size in config doc

* Add tokenizer files from MBart50

* Allow no lang code in regular tokenizer

* Add PLBart Tokenizer Converters

* Remove mask from multi tokenizer

* Remove mask from multi tokenizer

* Change from MBart-50 to MBart tokenizer

* Fix names and modify src/tgt behavior

* Fix imports for tokenizer

* Remove <mask> from multi tokenizer

* Fix style

* Change tokenizer_class to processor_class

* Add attribute map to config class

* Update modeling file to modified MBart code

* Update configuration file to MBart style configuration

* Fix tokenizer

* Separate tokenizers

* Fix error in tokenization auto

* Copy MBart tests

* Replace with MBart tokenization tests

* Fix style

* Fix language code in multi tokenizer

* Fix configuration docs

* Add entry for plbart_multi in transformers init

* Add dummy objects and fix imports

* Fix modeling tests

* Add TODO in config

* Fix copyright year

* Fix modeling docs and test

* Fix some tokenization tests and style

* Add changes from review

* Fix copies

* Fix docs

* Fix docs

* Fix style

* Fix year

* Add changes from review

* Remove extra changes

* Fix base tokenizer and doc

* Fix style

* Fix modeling and slow tokenizer tests

* Remove Multi-tokenizer Converter and Tests

* Delete QA model and Multi Tokenizer dummy objects

* Fix repo consistency and code quality issues

* Fix example documentation

* Fix style

* Remove PLBartTokenizer from type checking in init

* Fix consistency issue

* Add changes from review

* Fix style

* Remove PLBartTokenizerFast

* Remove FastTokenizer converter

* Fix AutoTokenzier mapping

* Add plbart to toctree and fix consistency issues

* Add language codes tokenizer test

* Fix styling and doc issues

* Add fixes for failing tests

* Fix copies

* Fix failing modeling test

* Change assert to assertTrue in modeling tests
2022-02-18 14:17:09 +01:00
..
test_module Allow custom code for Processors (#15649) 2022-02-15 09:44:35 -05:00
tf_ops Check TF ops for ONNX compliance (#10025) 2021-02-15 07:55:10 -05:00
check_copies.py Upgrade black to version ~=22.0 (#15565) 2022-02-09 09:28:57 -05:00
check_dummies.py Better dummies (#15148) 2022-01-14 10:59:41 -05:00
check_inits.py Ignore empty subfolders when identifying submodules (#15204) 2022-01-18 09:48:46 -05:00
check_repo.py Add PLBart (#13269) 2022-02-18 14:17:09 +01:00
check_table.py Add Tensorflow handling of ONNX conversion (#13831) 2022-02-10 11:18:41 +01:00
check_tf_ops.py Check TF ops for ONNX compliance (#10025) 2021-02-15 07:55:10 -05:00
custom_init_isort.py Sort init import (#10801) 2021-03-19 16:17:13 -04:00
documentation_tests.txt [DocTests Speech] Add doc tests for all speech models (#15031) 2022-01-27 14:29:31 +01:00
download_glue_data.py Raise exceptions instead of asserts (#13907) 2021-10-07 12:44:23 +05:30
get_modified_files.py Upgrade black to version ~=22.0 (#15565) 2022-02-09 09:28:57 -05:00
notification_service.py Nightly torch ci (#13550) 2021-09-13 16:17:29 -04:00
prepare_for_doc_test.py [DocTests Speech] Add doc tests for all speech models (#15031) 2022-01-27 14:29:31 +01:00
print_env_pt.py [CI] move env print to util, add pt, nccl versions (#14607) 2021-12-03 08:18:36 -05:00
release.py Docs for v4.14.0 2021-12-15 18:29:53 +01:00
style_doc.py Upgrade black to version ~=22.0 (#15565) 2022-02-09 09:28:57 -05:00
tests_fetcher.py Fix tests_fetcher (#15376) 2022-01-27 14:17:48 -05:00
update_metadata.py Replace commit sha by commit url for update jobs (#14852) 2021-12-21 11:17:11 -05:00