riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-31 02:02:21 +06:00
Code Issues Actions 33 Packages Projects Releases Wiki Activity
8,205 Commits 887 Branches 232 Tags 2.1 GiB
9f3aa46f45
Commit Graph

8205 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Patrick von Platen
9f3aa46f45
Add Unispeech & Unispeech-SAT (#13963) 
* unispeech

* add copy from

* remove hubert copy from

* finish for today

* add unispeech-sat

* adapt more

* up

* up

* up

* up

* add modeling

* add tests

* up

* up

* finish

* up

* Apply suggestions from code review

* up

* up

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* up

* up

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-10-26 18:59:58 +02:00
Patrick von Platen
9799f4e150
Update README.md 2021-10-26 18:59:25 +02:00
Stas Bekman
bfd8176636
[megatron_gpt2] dynamic gelu, add tokenizer, save config (#13928) 
* [megatron_gpt2] dynamic gelu, add tokenizer, save config

* cleanup

* Update src/transformers/models/megatron_gpt2/convert_megatron_gpt2_checkpoint.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* apply suggestions

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-10-26 09:09:54 -07:00
Sergio Valcarcel Macua
919a964b8f
Include Keras tensor in the allowed types (#14155) 
* Include KerasTensor in allowed types

- This allows propagating symbolic tensors through TFBert models and layers' call(),
  which allows converting the subclass models to functional models.

* Style pass

Co-authored-by: Sergio Valcarcel Macua <sergiov@graphcore.ai>
Co-authored-by: matt <rocketknight1@gmail.com>
 2021-10-26 15:08:59 +01:00
Patrick von Platen
f5ed19f57d
[Speech Recognition] - Distributed training: Make sure vocab file removal and creation don't interfer (#14161) 
* up

* better
 2021-10-26 15:59:33 +02:00
Yih-Dar
840fc8dbca
Add vision_encoder_decoder to models/__init__.py (#14151) 
* Add vision_encoder_decoder

* Update _ignore_modules in get_model_modules()

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2021-10-26 07:36:17 -04:00
Patrick von Platen
e248e9b042
up (#14154) 2021-10-26 13:08:18 +02:00
Thomas Chaigneau
1f60df81b2
Add Camembert to models exportable with ONNX (#14059) 
Add Camembert to models exportable with ONNX

Co-authored-by: Thomas.Chaigneau <thomas.chaigneau@arkea.com>
Co-authored-by: Michael Benayoun <mickbenayoun@gmail.com>
 2021-10-26 11:22:22 +02:00
Patrick von Platen
0c3174c758
Add TF<>PT and Flax<>PT everywhere (#14047) 
* up

* up

* up

* up

* up

* up

* up

* add clip

* fix clip PyTorch

* fix clip PyTorch

* up

* up

* up

* up

* up

* up

* up
 2021-10-25 23:55:08 +02:00
Sylvain Gugger
8560b55b5e
Fix lazy init to stop hiding errors in import (#14124) 2021-10-25 16:53:47 -04:00
Patrick von Platen
c99a2832ed
Update README.md 2021-10-25 19:50:36 +02:00
Patrick von Platen
1a9381c60d
Update README.md 2021-10-25 19:49:51 +02:00
Matt
3e8761ab80
Enable DefaultDataCollator class (#14141) 2021-10-25 15:04:54 +01:00
Matt
84b9579da7
Remove unneeded to_tensor() in TF inline example (#14140) 2021-10-25 15:04:36 +01:00
Chi-Liang, Liu
1967c43eb9
BartEnocder add set_input_embeddings (#13960) 
* BartEnocder add set_input_embeddings

To unify the interface, add set_input_embeddings to BartEncoder.

* BartEnocder add get_input_embeddings
 2021-10-25 13:58:29 +02:00
Reza Gharibi
3e04a41a9b
Fix some writing issues in the docs (#14136) 
* Fix some writing issues in the docs

* Run code quality check
 2021-10-25 07:48:02 -04:00
Reza Gharibi
2ac65551ea
Fix rendering of examples version links (#14134) 2021-10-25 07:45:44 -04:00
karthikrangasai
1b871e091b
Supporting Seq2Seq model for question answering task (#13432) 
* Add seq2seq example for QnA on SQuAD Dataset.

* Changes from review - Fixing styling mistakes.

* Added how to example in README, simplified the access to dataset's preprocess function.

* Added tests for the seq2seq QA example.

* Change dataset column name to fix tests.

* Fix test command mistake.

* Add missing argument 'ignore_pad_token_for_loss' from DataTrainingArguments.

* Add missing argument 'num_beams' from DataTrainingArguments.

* Fix processing of output predicted token ids so that tokenizer decode gets appropriate input. Updated assertion conditions on the tests.
 2021-10-25 07:42:53 -04:00
Reza Gharibi
6b83090e80
Fix some typos in the docs (#14126) 
* Fix some typos in the docs

* Fix a styling issue

* Fix code quality check error
 2021-10-25 07:40:44 -04:00
Kevin Ko
95bab53868
Update TP parallel GEMM image (#14112) 
* Update TP parallel GEMM image

* Delete parallelism-tp-parallel_gemm.png

* Update parallelism-tp-parallel_gemm.png
 2021-10-22 12:57:48 -07:00
Li-Huai (Allan) Lin
62ccbe0960
Rename variables with unclear naming (#14122) 
* Rename var

* Add comments
 2021-10-22 19:05:45 +02:00
Antonio Carlos Falcão Petri
05a2afc252
Add missing --validation_split_percentage data args (#14119) 2021-10-22 19:04:54 +02:00
Baizhou Huang
c7ccb2e779
Fix assertion in models (#14090) 
* replace assertions in src/transformers/models/luke/convert_luke_original_pytorch_checkpoint_to_pytorch.py

* replace assertions in src/transformers/models/marian/convert_marian_to_pytorch.py

* Update src/transformers/models/luke/convert_luke_original_pytorch_checkpoint_to_pytorch.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/marian/convert_marian_to_pytorch.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/marian/convert_marian_to_pytorch.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/marian/convert_marian_to_pytorch.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/marian/convert_marian_to_pytorch.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/marian/convert_marian_to_pytorch.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/marian/convert_marian_to_pytorch.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/marian/convert_marian_to_pytorch.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/marian/convert_marian_to_pytorch.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: skpig <1900012999@pku.edu.cn>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-10-22 10:03:09 -04:00
Sylvain Gugger
16d7b70b80 Update Korean README to master 2021-10-22 08:13:04 -04:00
Jayesh Dewangan
fa4abdb3ea
Replace assertions with valueError Exeptions (#14117) 
* Replace assertions with valueError Exeptions

* Reformatted
 2021-10-22 07:45:32 -04:00
Yeoun Yi
9f53f049c6
Translate README.md to Korean (#14015) 
* Create README_ko.md

* Update README.md

* Update README_zh-hans.md

* Update README_zh-hant.md

* Update README_ko.md

* Update check_copies.py

* Update README_ko.md

* typo

* match with readme_ko
 2021-10-22 07:42:31 -04:00
David del Río Medina
f5a49bfa4d
Replace assert statements with exceptions (#13871) (#13901) 
* Replace assert statements with exceptions (#13871)

* Change f-strings when not needed (flake8)

* Replace assert statements with exceptions (#13871)

* Change f-strings when not needed (flake8)

* Improve error message as suggested by reviewer

* Fix identation bug

* Fix style errors
 2021-10-22 13:11:40 +02:00
Patrick von Platen
70f186f61e
up (#14116) 2021-10-22 11:01:26 +02:00
Deepanshu verma
ca2ef7dfcd
Changed asserts to ValueError (#14091) 2021-10-21 18:07:18 -04:00
Reza Gharibi
7888914edd
Fix a typo in preprocessing docs (#14108) 2021-10-21 17:00:26 -04:00
lee1jun
d432a654f6
fix typo in license docstring (#14094) 
last line: "# limitations under the License." is missing
 2021-10-21 15:31:32 -04:00
David del Río Medina
7af55d3a1c
Replace assertion with ValueError exception (#14098) 2021-10-21 15:31:00 -04:00
stalkermustang
f00bceab8d
Fix typo in comment (#14102) 2021-10-21 15:29:17 -04:00
Li-Huai (Allan) Lin
234cfefbb0
Fix ignore_mismatched_sizes (#14085) 
* Fix

* Style

* Name

* Fix tests

* Style

* Remove embed sizes checking

* Disable some tests

* Fix

* Apply suggestion
 2021-10-21 12:31:29 -04:00
Anton Lozhkov
e03544a138
[Examples] Add audio classification notebooks (#14099) 
* Update SEW integration test tolerance

* Add audio classification notebooks
 2021-10-21 19:15:46 +03:00
Sylvain Gugger
0f502682fb Pin PyTorch to make CI green 2021-10-21 11:59:23 -04:00
Christopher Akiki
f9c16b02e3
Replace "Masked" with "Causal" in TF CLM example (#14014) 2021-10-21 16:19:30 +01:00
David del Río Medina
3187228206
Replace assertions with ValueError exceptions (#14061) 
* Replace assertions with ValueError exceptions

* Format error messages as suggested
 2021-10-21 07:32:27 -04:00
Weston King-Leatham
9e4ea25175
Change asserts in src/transformers/models/xlnet/ to raise ValueError (#14088) 
* Change asserts in src/transformers/models/xlnet/ to raise ValueError

* Update src/transformers/models/xlnet/modeling_tf_xlnet.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-10-21 07:27:32 -04:00
Patrick von Platen
e9d2a639f4
up (#14093) 2021-10-21 10:30:02 +02:00
Reza Gharibi
49155d2431
Fix broken link in translation section (#14087) 2021-10-20 15:10:57 -04:00
Leandro von Werra
0270d44f57
Context managers (#13900) 
* add `ContextManagers` for lists of contexts

* fix import sorting

* add `ContextManagers` tests
 2021-10-20 14:15:47 +02:00
Sylvain Gugger
f875fb0e5f
Fix label attribution in token classification examples (#14055) 2021-10-20 07:55:14 -04:00
Baizhou Huang
31560f6397
Fix assert in src/transformers/data/datasets/language_modeling.py (#14077) 
* replace assertion with ValueError

* fix code style

Co-authored-by: skpig <1900012999@pku.edu.cn>
 2021-10-20 07:54:39 -04:00
Kwanghee Choi
0106826a65
Fix missing autocast() in Trainer.prediction_step() (#14075) 
Co-authored-by: jonas <jonas@hpcnt.com>
 2021-10-20 07:51:30 -04:00
Baizhou Huang
a43d9352a9
replace assert with exception in src/transformers/utils/model_pararallel_utils.py (#14072) 
* replace assert with exception in src/transformers/utils/model_parallel_utils.py

* fix some code style

* fix typo

Co-authored-by: skpig <1900012999@pku.edu.cn>
 2021-10-20 07:43:45 -04:00
Patrick von Platen
53dc39d821
up (#14079) 2021-10-20 13:01:42 +02:00
Patrick von Platen
0bc2e54f00
Add ASR colabs (#14067) 
* up

* Update notebooks/README.md
 2021-10-20 11:51:41 +02:00
Anton Lozhkov
dbaf49203e
[Examples] Use Audio feature in speech classification (#14052) 
* Update SEW integration test tolerance

* Update audio classification

* Update test

* Remove torchaudio

* Add dataset revision

* Hub branch naming

* Revert dataset revisions

* Update datasets
 2021-10-20 12:22:43 +03:00
Robert Stone
3fefa292c1
Trainer._load_rng_state() path fix (#14069) (#14071) 2021-10-19 22:06:19 -04:00