riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-31 02:02:21 +06:00
Code Issues Actions 23 Packages Projects Releases Wiki Activity
8,195 Commits 887 Branches 232 Tags 2.1 GiB
c99a2832ed
Commit Graph

8195 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Patrick von Platen
c99a2832ed
Update README.md 2021-10-25 19:50:36 +02:00
Patrick von Platen
1a9381c60d
Update README.md 2021-10-25 19:49:51 +02:00
Matt
3e8761ab80
Enable DefaultDataCollator class (#14141) 2021-10-25 15:04:54 +01:00
Matt
84b9579da7
Remove unneeded to_tensor() in TF inline example (#14140) 2021-10-25 15:04:36 +01:00
Chi-Liang, Liu
1967c43eb9
BartEnocder add set_input_embeddings (#13960) 
* BartEnocder add set_input_embeddings

To unify the interface, add set_input_embeddings to BartEncoder.

* BartEnocder add get_input_embeddings
 2021-10-25 13:58:29 +02:00
Reza Gharibi
3e04a41a9b
Fix some writing issues in the docs (#14136) 
* Fix some writing issues in the docs

* Run code quality check
 2021-10-25 07:48:02 -04:00
Reza Gharibi
2ac65551ea
Fix rendering of examples version links (#14134) 2021-10-25 07:45:44 -04:00
karthikrangasai
1b871e091b
Supporting Seq2Seq model for question answering task (#13432) 
* Add seq2seq example for QnA on SQuAD Dataset.

* Changes from review - Fixing styling mistakes.

* Added how to example in README, simplified the access to dataset's preprocess function.

* Added tests for the seq2seq QA example.

* Change dataset column name to fix tests.

* Fix test command mistake.

* Add missing argument 'ignore_pad_token_for_loss' from DataTrainingArguments.

* Add missing argument 'num_beams' from DataTrainingArguments.

* Fix processing of output predicted token ids so that tokenizer decode gets appropriate input. Updated assertion conditions on the tests.
 2021-10-25 07:42:53 -04:00
Reza Gharibi
6b83090e80
Fix some typos in the docs (#14126) 
* Fix some typos in the docs

* Fix a styling issue

* Fix code quality check error
 2021-10-25 07:40:44 -04:00
Kevin Ko
95bab53868
Update TP parallel GEMM image (#14112) 
* Update TP parallel GEMM image

* Delete parallelism-tp-parallel_gemm.png

* Update parallelism-tp-parallel_gemm.png
 2021-10-22 12:57:48 -07:00
Li-Huai (Allan) Lin
62ccbe0960
Rename variables with unclear naming (#14122) 
* Rename var

* Add comments
 2021-10-22 19:05:45 +02:00
Antonio Carlos Falcão Petri
05a2afc252
Add missing --validation_split_percentage data args (#14119) 2021-10-22 19:04:54 +02:00
Baizhou Huang
c7ccb2e779
Fix assertion in models (#14090) 
* replace assertions in src/transformers/models/luke/convert_luke_original_pytorch_checkpoint_to_pytorch.py

* replace assertions in src/transformers/models/marian/convert_marian_to_pytorch.py

* Update src/transformers/models/luke/convert_luke_original_pytorch_checkpoint_to_pytorch.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/marian/convert_marian_to_pytorch.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/marian/convert_marian_to_pytorch.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/marian/convert_marian_to_pytorch.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/marian/convert_marian_to_pytorch.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/marian/convert_marian_to_pytorch.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/marian/convert_marian_to_pytorch.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/marian/convert_marian_to_pytorch.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/marian/convert_marian_to_pytorch.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: skpig <1900012999@pku.edu.cn>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-10-22 10:03:09 -04:00
Sylvain Gugger
16d7b70b80 Update Korean README to master 2021-10-22 08:13:04 -04:00
Jayesh Dewangan
fa4abdb3ea
Replace assertions with valueError Exeptions (#14117) 
* Replace assertions with valueError Exeptions

* Reformatted
 2021-10-22 07:45:32 -04:00
Yeoun Yi
9f53f049c6
Translate README.md to Korean (#14015) 
* Create README_ko.md

* Update README.md

* Update README_zh-hans.md

* Update README_zh-hant.md

* Update README_ko.md

* Update check_copies.py

* Update README_ko.md

* typo

* match with readme_ko
 2021-10-22 07:42:31 -04:00
David del Río Medina
f5a49bfa4d
Replace assert statements with exceptions (#13871) (#13901) 
* Replace assert statements with exceptions (#13871)

* Change f-strings when not needed (flake8)

* Replace assert statements with exceptions (#13871)

* Change f-strings when not needed (flake8)

* Improve error message as suggested by reviewer

* Fix identation bug

* Fix style errors
 2021-10-22 13:11:40 +02:00
Patrick von Platen
70f186f61e
up (#14116) 2021-10-22 11:01:26 +02:00
Deepanshu verma
ca2ef7dfcd
Changed asserts to ValueError (#14091) 2021-10-21 18:07:18 -04:00
Reza Gharibi
7888914edd
Fix a typo in preprocessing docs (#14108) 2021-10-21 17:00:26 -04:00
lee1jun
d432a654f6
fix typo in license docstring (#14094) 
last line: "# limitations under the License." is missing
 2021-10-21 15:31:32 -04:00
David del Río Medina
7af55d3a1c
Replace assertion with ValueError exception (#14098) 2021-10-21 15:31:00 -04:00
stalkermustang
f00bceab8d
Fix typo in comment (#14102) 2021-10-21 15:29:17 -04:00
Li-Huai (Allan) Lin
234cfefbb0
Fix ignore_mismatched_sizes (#14085) 
* Fix

* Style

* Name

* Fix tests

* Style

* Remove embed sizes checking

* Disable some tests

* Fix

* Apply suggestion
 2021-10-21 12:31:29 -04:00
Anton Lozhkov
e03544a138
[Examples] Add audio classification notebooks (#14099) 
* Update SEW integration test tolerance

* Add audio classification notebooks
 2021-10-21 19:15:46 +03:00
Sylvain Gugger
0f502682fb Pin PyTorch to make CI green 2021-10-21 11:59:23 -04:00
Christopher Akiki
f9c16b02e3
Replace "Masked" with "Causal" in TF CLM example (#14014) 2021-10-21 16:19:30 +01:00
David del Río Medina
3187228206
Replace assertions with ValueError exceptions (#14061) 
* Replace assertions with ValueError exceptions

* Format error messages as suggested
 2021-10-21 07:32:27 -04:00
Weston King-Leatham
9e4ea25175
Change asserts in src/transformers/models/xlnet/ to raise ValueError (#14088) 
* Change asserts in src/transformers/models/xlnet/ to raise ValueError

* Update src/transformers/models/xlnet/modeling_tf_xlnet.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-10-21 07:27:32 -04:00
Patrick von Platen
e9d2a639f4
up (#14093) 2021-10-21 10:30:02 +02:00
Reza Gharibi
49155d2431
Fix broken link in translation section (#14087) 2021-10-20 15:10:57 -04:00
Leandro von Werra
0270d44f57
Context managers (#13900) 
* add `ContextManagers` for lists of contexts

* fix import sorting

* add `ContextManagers` tests
 2021-10-20 14:15:47 +02:00
Sylvain Gugger
f875fb0e5f
Fix label attribution in token classification examples (#14055) 2021-10-20 07:55:14 -04:00
Baizhou Huang
31560f6397
Fix assert in src/transformers/data/datasets/language_modeling.py (#14077) 
* replace assertion with ValueError

* fix code style

Co-authored-by: skpig <1900012999@pku.edu.cn>
 2021-10-20 07:54:39 -04:00
Kwanghee Choi
0106826a65
Fix missing autocast() in Trainer.prediction_step() (#14075) 
Co-authored-by: jonas <jonas@hpcnt.com>
 2021-10-20 07:51:30 -04:00
Baizhou Huang
a43d9352a9
replace assert with exception in src/transformers/utils/model_pararallel_utils.py (#14072) 
* replace assert with exception in src/transformers/utils/model_parallel_utils.py

* fix some code style

* fix typo

Co-authored-by: skpig <1900012999@pku.edu.cn>
 2021-10-20 07:43:45 -04:00
Patrick von Platen
53dc39d821
up (#14079) 2021-10-20 13:01:42 +02:00
Patrick von Platen
0bc2e54f00
Add ASR colabs (#14067) 
* up

* Update notebooks/README.md
 2021-10-20 11:51:41 +02:00
Anton Lozhkov
dbaf49203e
[Examples] Use Audio feature in speech classification (#14052) 
* Update SEW integration test tolerance

* Update audio classification

* Update test

* Remove torchaudio

* Add dataset revision

* Hub branch naming

* Revert dataset revisions

* Update datasets
 2021-10-20 12:22:43 +03:00
Robert Stone
3fefa292c1
Trainer._load_rng_state() path fix (#14069) (#14071) 2021-10-19 22:06:19 -04:00
Meng Zhou
3892d09f4f
update to_py_obj to support np.number (#14064) 
Co-authored-by: 眸浩 <mouhao.zm@alibaba-inc.com>
 2021-10-19 14:30:53 -04:00
Pedro Marques
122c2f81b7
TF Model train and eval step metrics for seq2seq models. (#14009) 
* TF Model train and eval step metrics for seq2seq models.

When using a model with a seq2seq output compute metrics against logits.

* Removing vestigial code

Co-authored-by: matt <rocketknight1@gmail.com>
 2021-10-19 12:14:21 +01:00
Thomas Wang
fde4867f97
Fix passing None as concrete args (#14022) 2021-10-19 10:56:17 +02:00
Ihor Omelchenko
9eda0d156d
Fix typo (#14056) 2021-10-18 18:03:39 -04:00
Weizhe Yuan
7a3147e9b8
fix typo (#14049) 2021-10-18 18:03:11 -04:00
Patrick von Platen
d5ff69fce9
[Speech] Refactor Examples (#14040) 
* adapt_examples

* up

* up

* up

* up

* add auto models

* finish
 2021-10-18 17:43:35 +02:00
Sylvain Gugger
2024faf171
Fix save when laod_best_model_at_end=True (#14054) 2021-10-18 10:22:57 -04:00
Sylvain Gugger
2c60ff2fe2
Add an API to register objects to Auto classes (#13989) 
* Add API to register a new object in auto classes

* Fix test

* Documentation

* Add to tokenizers and test

* Add cleanup after tests

* Be more careful

* Move import

* Move import

* Cleanup in TF test too

* Add consistency check

* Add documentation

* Style

* Update docs/source/model_doc/auto.rst

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Update src/transformers/models/auto/auto_factory.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
 2021-10-18 10:22:46 -04:00
Dat Quoc Nguyen
3d587c5343
Add BARTpho: Pre-trained Sequence-to-Sequence Models for Vietnamese (#13788) 
* Add the pre-trained BARTpho model

* Add the pre-trained BARTpho model

* Add the pre-trained BARTpho model

* Fix incorrectly sorted and/or formatted imports

* Fix incorrectly sorted and/or formatted style

* Fix check_dummies

* Fix check_dummies

* Fix check_dummies

* Update docs/source/model_doc/bartpho.rst

Co-authored-by: Suraj Patil <surajp815@gmail.com>

* Update src/transformers/models/bartpho/__init__.py

Co-authored-by: Suraj Patil <surajp815@gmail.com>

* Update src/transformers/models/bartpho/tokenization_bartpho.py

Co-authored-by: Suraj Patil <surajp815@gmail.com>

* Update tests/test_tokenization_bartpho.py

Co-authored-by: Suraj Patil <surajp815@gmail.com>

* Update src/transformers/models/bartpho/tokenization_bartpho.py

Co-authored-by: Suraj Patil <surajp815@gmail.com>

* Update tests/test_tokenization_bartpho.py

Co-authored-by: Suraj Patil <surajp815@gmail.com>

* Update docs/source/model_doc/bartpho.rst

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update docs/source/model_doc/bartpho.rst

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/bartpho/__init__.py

Co-authored-by: Suraj Patil <surajp815@gmail.com>

* Add the pre-trained BARTpho model

* Add Tips section in doc and details of monolingual_vocab_file

* Fix conflicts

* Add another tip related to monolingual_vocab_file

* Readd dependency_versions_table.py

* Handle failing checks

* Remove test_list.txt

* Remove md5sum.saved

* Revise Readme.md

Co-authored-by: Suraj Patil <surajp815@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-10-18 10:16:46 -04:00
Patrick von Platen
7c6cd0ac28
up (#14046) 2021-10-18 12:59:18 +02:00