riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-20 13:08:21 +06:00
Code Issues Actions 7 Packages Projects Releases Wiki Activity
8,821 Commits 876 Branches 227 Tags 2 GiB
7d45a2e81c
Commit Graph

8821 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Stas Bekman
123cce6ffc
[modeling_utils] respect original dtype in _get_resized_lm_head (#14181) 
* respect dtype in _get_resized_lm_head

* Update src/transformers/modeling_utils.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* consistency

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-10-27 19:01:50 -07:00
Patrick von Platen
88cd82e801
Update README.md 2021-10-28 02:35:01 +02:00
Patrick von Platen
e118db15d6
Update README.md 2021-10-28 01:59:27 +02:00
Patrick von Platen
01b1466983
[TPU tests] Enable first TPU examples pytorch (#14121) 
* up

* up

* fix

* up

* Update examples/pytorch/test_xla_examples.py

* correct labels

* up

* up

* up

* up

* up

* up
 2021-10-28 01:22:28 +02:00
Anton Lozhkov
232822f36d
Add DistilHuBERT (#14174) 
* Add conversion

* Rename

* Add an integration test and remove layer_norm

* Remove layer_norm from the converter

* wording

* Fix imports
 2021-10-27 20:17:31 +03:00
Lahfa Samy
e5b8ffb848
Replace assert of data/data_collator.py by ValueError (#14131) 
* Replace assert of data_collator.py by ValueError

* Replace assert of data_collator.py by ValueError
 2021-10-27 12:19:10 -04:00
Anton Lozhkov
25ceb81871
[Pipelines] Fix ASR model types check (#14178) 2021-10-27 17:17:47 +03:00
Patrick von Platen
6200fd7bbc
[Gradient checkpointing] Enable for Deberta + DebertaV2 + SEW-D (#14175) 
* up

* up

* finish

* up

* final changes
 2021-10-27 15:47:20 +02:00
Anton Lozhkov
e1dc5afd28
Add SEW CTC models (#14158) 
* Add SEW CTC models

* Update paths

* Update paths
 2021-10-27 12:21:09 +03:00
Lysandre Debut
1e53faeb2e
Fix gelu test for torch 1.10 (#14167) 2021-10-26 22:20:51 -04:00
Kamal Raj
8ddbfe9752
switch to inference_mode from no_gard (#13667) 
* switch to inference_mode from no_gard
faster inference

* added switch to support older version of pytorch
 2021-10-26 18:02:58 -04:00
Emanuel Huber
ebd48c6de5
Replace assertions with ValueError exception (#14142) 
Updated masked-language modeling examples in pytorch
with convention defined by #12789
 2021-10-26 17:14:29 -04:00
Matthew Goldey
42bfb83d74
fix typos in error messages in speech recognition example and modelcard.py (#14166) 
* specify the text column name in the error message

* pluralize the word fields
 2021-10-26 16:36:26 -04:00
Jangwon Park
41dad89f70
chore: typo on ner accelerate example code (#14150) 2021-10-26 16:23:41 -04:00
Lysandre
27c888db6c Fix copies 2021-10-26 15:48:28 -04:00
Jay Zhang
3f23634a17
[ONNX] Add symbolic function for XSoftmax op for exporting to ONNX. (#14013) 
* Add symbolic function for XSoftmax op for exporting to ONNX.

* Fix format issues.

* Fix a CI issue relative to copies.
 2021-10-26 15:25:02 -04:00
Patrick von Platen
9f3aa46f45
Add Unispeech & Unispeech-SAT (#13963) 
* unispeech

* add copy from

* remove hubert copy from

* finish for today

* add unispeech-sat

* adapt more

* up

* up

* up

* up

* add modeling

* add tests

* up

* up

* finish

* up

* Apply suggestions from code review

* up

* up

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* up

* up

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-10-26 18:59:58 +02:00
Patrick von Platen
9799f4e150
Update README.md 2021-10-26 18:59:25 +02:00
Stas Bekman
bfd8176636
[megatron_gpt2] dynamic gelu, add tokenizer, save config (#13928) 
* [megatron_gpt2] dynamic gelu, add tokenizer, save config

* cleanup

* Update src/transformers/models/megatron_gpt2/convert_megatron_gpt2_checkpoint.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* apply suggestions

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-10-26 09:09:54 -07:00
Sergio Valcarcel Macua
919a964b8f
Include Keras tensor in the allowed types (#14155) 
* Include KerasTensor in allowed types

- This allows propagating symbolic tensors through TFBert models and layers' call(),
  which allows converting the subclass models to functional models.

* Style pass

Co-authored-by: Sergio Valcarcel Macua <sergiov@graphcore.ai>
Co-authored-by: matt <rocketknight1@gmail.com>
 2021-10-26 15:08:59 +01:00
Patrick von Platen
f5ed19f57d
[Speech Recognition] - Distributed training: Make sure vocab file removal and creation don't interfer (#14161) 
* up

* better
 2021-10-26 15:59:33 +02:00
Yih-Dar
840fc8dbca
Add vision_encoder_decoder to models/__init__.py (#14151) 
* Add vision_encoder_decoder

* Update _ignore_modules in get_model_modules()

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2021-10-26 07:36:17 -04:00
Patrick von Platen
e248e9b042
up (#14154) 2021-10-26 13:08:18 +02:00
Thomas Chaigneau
1f60df81b2
Add Camembert to models exportable with ONNX (#14059) 
Add Camembert to models exportable with ONNX

Co-authored-by: Thomas.Chaigneau <thomas.chaigneau@arkea.com>
Co-authored-by: Michael Benayoun <mickbenayoun@gmail.com>
 2021-10-26 11:22:22 +02:00
Patrick von Platen
0c3174c758
Add TF<>PT and Flax<>PT everywhere (#14047) 
* up

* up

* up

* up

* up

* up

* up

* add clip

* fix clip PyTorch

* fix clip PyTorch

* up

* up

* up

* up

* up

* up

* up
 2021-10-25 23:55:08 +02:00
Sylvain Gugger
8560b55b5e
Fix lazy init to stop hiding errors in import (#14124) 2021-10-25 16:53:47 -04:00
Patrick von Platen
c99a2832ed
Update README.md 2021-10-25 19:50:36 +02:00
Patrick von Platen
1a9381c60d
Update README.md 2021-10-25 19:49:51 +02:00
Matt
3e8761ab80
Enable DefaultDataCollator class (#14141) 2021-10-25 15:04:54 +01:00
Matt
84b9579da7
Remove unneeded to_tensor() in TF inline example (#14140) 2021-10-25 15:04:36 +01:00
Chi-Liang, Liu
1967c43eb9
BartEnocder add set_input_embeddings (#13960) 
* BartEnocder add set_input_embeddings

To unify the interface, add set_input_embeddings to BartEncoder.

* BartEnocder add get_input_embeddings
 2021-10-25 13:58:29 +02:00
Reza Gharibi
3e04a41a9b
Fix some writing issues in the docs (#14136) 
* Fix some writing issues in the docs

* Run code quality check
 2021-10-25 07:48:02 -04:00
Reza Gharibi
2ac65551ea
Fix rendering of examples version links (#14134) 2021-10-25 07:45:44 -04:00
karthikrangasai
1b871e091b
Supporting Seq2Seq model for question answering task (#13432) 
* Add seq2seq example for QnA on SQuAD Dataset.

* Changes from review - Fixing styling mistakes.

* Added how to example in README, simplified the access to dataset's preprocess function.

* Added tests for the seq2seq QA example.

* Change dataset column name to fix tests.

* Fix test command mistake.

* Add missing argument 'ignore_pad_token_for_loss' from DataTrainingArguments.

* Add missing argument 'num_beams' from DataTrainingArguments.

* Fix processing of output predicted token ids so that tokenizer decode gets appropriate input. Updated assertion conditions on the tests.
 2021-10-25 07:42:53 -04:00
Reza Gharibi
6b83090e80
Fix some typos in the docs (#14126) 
* Fix some typos in the docs

* Fix a styling issue

* Fix code quality check error
 2021-10-25 07:40:44 -04:00
Kevin Ko
95bab53868
Update TP parallel GEMM image (#14112) 
* Update TP parallel GEMM image

* Delete parallelism-tp-parallel_gemm.png

* Update parallelism-tp-parallel_gemm.png
 2021-10-22 12:57:48 -07:00
Li-Huai (Allan) Lin
62ccbe0960
Rename variables with unclear naming (#14122) 
* Rename var

* Add comments
 2021-10-22 19:05:45 +02:00
Antonio Carlos Falcão Petri
05a2afc252
Add missing --validation_split_percentage data args (#14119) 2021-10-22 19:04:54 +02:00
Baizhou Huang
c7ccb2e779
Fix assertion in models (#14090) 
* replace assertions in src/transformers/models/luke/convert_luke_original_pytorch_checkpoint_to_pytorch.py

* replace assertions in src/transformers/models/marian/convert_marian_to_pytorch.py

* Update src/transformers/models/luke/convert_luke_original_pytorch_checkpoint_to_pytorch.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/marian/convert_marian_to_pytorch.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/marian/convert_marian_to_pytorch.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/marian/convert_marian_to_pytorch.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/marian/convert_marian_to_pytorch.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/marian/convert_marian_to_pytorch.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/marian/convert_marian_to_pytorch.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/marian/convert_marian_to_pytorch.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/marian/convert_marian_to_pytorch.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: skpig <1900012999@pku.edu.cn>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-10-22 10:03:09 -04:00
Sylvain Gugger
16d7b70b80 Update Korean README to master 2021-10-22 08:13:04 -04:00
Jayesh Dewangan
fa4abdb3ea
Replace assertions with valueError Exeptions (#14117) 
* Replace assertions with valueError Exeptions

* Reformatted
 2021-10-22 07:45:32 -04:00
Yeoun Yi
9f53f049c6
Translate README.md to Korean (#14015) 
* Create README_ko.md

* Update README.md

* Update README_zh-hans.md

* Update README_zh-hant.md

* Update README_ko.md

* Update check_copies.py

* Update README_ko.md

* typo

* match with readme_ko
 2021-10-22 07:42:31 -04:00
David del Río Medina
f5a49bfa4d
Replace assert statements with exceptions (#13871) (#13901) 
* Replace assert statements with exceptions (#13871)

* Change f-strings when not needed (flake8)

* Replace assert statements with exceptions (#13871)

* Change f-strings when not needed (flake8)

* Improve error message as suggested by reviewer

* Fix identation bug

* Fix style errors
 2021-10-22 13:11:40 +02:00
Patrick von Platen
70f186f61e
up (#14116) 2021-10-22 11:01:26 +02:00
Deepanshu verma
ca2ef7dfcd
Changed asserts to ValueError (#14091) 2021-10-21 18:07:18 -04:00
Reza Gharibi
7888914edd
Fix a typo in preprocessing docs (#14108) 2021-10-21 17:00:26 -04:00
lee1jun
d432a654f6
fix typo in license docstring (#14094) 
last line: "# limitations under the License." is missing
 2021-10-21 15:31:32 -04:00
David del Río Medina
7af55d3a1c
Replace assertion with ValueError exception (#14098) 2021-10-21 15:31:00 -04:00
stalkermustang
f00bceab8d
Fix typo in comment (#14102) 2021-10-21 15:29:17 -04:00
Li-Huai (Allan) Lin
234cfefbb0
Fix ignore_mismatched_sizes (#14085) 
* Fix

* Style

* Name

* Fix tests

* Style

* Remove embed sizes checking

* Disable some tests

* Fix

* Apply suggestion
 2021-10-21 12:31:29 -04:00