transformers/docs/source
TomLim 1bd604d11c
[WIP] Add Tokenizer for MyT5 Model (#31286)
* Initial commit for MyT5 model

* custom implementation of MyT5 tokenizer, unused files deleted

* unittest for myt5 tokenizer

* upadate of import structure and style

* removed remmanents of MyT5Config

* fixed docstrings

* Updates after review: filled documentaion file, new docstrings and tests added

* Fixed code style issues

* fixed copied from to refer to function

* updated loading myt5 tokenizer in tests, added sample byte map file to fixtures

* changes after review

* removed redundant copied from

* removed redundant copied from

* optimalization and loading model from hf

* [run_slow] myt5

* [run-slow] myt5

* Updated en documentation for myt5

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

---------

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
2024-10-06 10:33:16 +02:00
..
ar [i18n-ar] Add File : docs/source/ar/_toctree.yml (#32696) 2024-09-16 10:02:03 -07:00
de Skip tests properly (#31308) 2024-06-26 21:59:08 +01:00
en [WIP] Add Tokenizer for MyT5 Model (#31286) 2024-10-06 10:33:16 +02:00
es Trainer - deprecate tokenizer for processing_class (#32385) 2024-10-02 14:08:46 +01:00
fr Trainer - deprecate tokenizer for processing_class (#32385) 2024-10-02 14:08:46 +01:00
hi More fixes for doctest (#30265) 2024-04-16 11:58:55 +02:00
it Updated the custom_models.md changed cross_entropy code (#33118) 2024-08-26 13:15:43 +02:00
ja Trainer - deprecate tokenizer for processing_class (#32385) 2024-10-02 14:08:46 +01:00
ko Trainer - deprecate tokenizer for processing_class (#32385) 2024-10-02 14:08:46 +01:00
ms Remove old TF port docs (#30426) 2024-04-23 16:06:20 +01:00
pt Trainer - deprecate tokenizer for processing_class (#32385) 2024-10-02 14:08:46 +01:00
te Trainer - deprecate tokenizer for processing_class (#32385) 2024-10-02 14:08:46 +01:00
tr Translate index.md to Turkish (#27093) 2023-11-08 08:35:20 -05:00
zh Trainer - deprecate tokenizer for processing_class (#32385) 2024-10-02 14:08:46 +01:00
_config.py [#29174] ImportError Fix: Trainer with PyTorch requires accelerate>=0.20.1 Fix (#29888) 2024-04-08 14:21:16 +01:00