* preserve type of `additional_special_tokens` in `special_token_map`
* format
* Update src/transformers/tokenization_utils_base.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Add README_zh-tw.md
* Add links to each README.
* Fix a mismatched term.
* Minor improvements.
* Rename language code to be more inclusive.
* Polish terms to make them fluent.
* Remove redundant spaces.
* Fix typo.
* Base test
* More test
* Fix mistake
* Add a docstring change
* Add doc ignore
* Add changes
* Add recursive dep search
* Add recursive dep search
* save
* Finalize test mapping
* Fix bug
* Print prettier
* Ignore comments and empty lines
* Make script runnable from anywhere
* Need dev install
* Like that
* Adapt
* Add as artifact
* Try on torch tests
* Fix yaml error
* Install GitPython
* Apply everywhere
* Be more defensive
* Revert to all tests if something is wrong
* Install GitPython
* Test if there are tests before launching.
* Fixes
* Fixes
* Fixes
* Fixes
* Bash syntax is horrible
* Be less stupid
* Try differently
* Typo
* Typo
* Typo
* Style
* Better name
* Escape quotes
* Ignore black unhelpful re-formatting
* Not a docstring
* Deal with inits in dependency map
* Run all tests once PR is merged.
* Add last job
* Apply suggestions from code review
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
* Stronger dependencies gather
* Ignore empty lines too!
* Clean up
* Fix quality
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
* We need to provide mask_time_indices to `_mask_hidden_states` to avoid applying the mask two times
* apply the same to wav2vec2
* Uniformize the style between hubert and wav2vec2
* fix tf as well
Co-authored-by: patrickvonplaten <patrick.v.platen@gmail.com>
* Adding TF translation example
* Fixes and style pass for TF translation example
* Remove unused postprocess_text copied from run_summarization
* Adding README
* Review fixes
* Move changes to model.config to after we've initialized the model
* fix_torch_device_generate_test
* remove @
* correct greedy search
* save intertmed
* add final logits bias
* correct
* up
* add more tests
* fix another bug
* finish tests
* finish marian tests
* up
Co-authored-by: Patrick von Platen <patrick@huggingface.co>
* change deprecation to be easily actionable
* Update src/transformers/tokenization_utils_base.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* rework as suggested
* one warning together
* fix format
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Add option to load a pretrained model with mismatched shapes
* Fail at loading when mismatched shapes in Flax
* Fix tests
* Update src/transformers/modeling_flax_utils.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* Address review comments
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* Wrong model is used, should be character instead of subword
In the original Google repo for CANINE there was mixup in the model names in the README.md, which was fixed 2 weeks ago. Since this transformer model was created before, it probably resulted in wrong use in this example.
s = subword, c = character
* canine.rst style fix
* Update docs/source/model_doc/canine.rst
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Styling canine.rst
* Added links to model cards.
* Fixed links to model cards.
Co-authored-by: Jeroen Steggink <978411+jsteggink@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
