riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-24 06:48:58 +06:00
Code Issues Actions 22 Packages Projects Releases Wiki Activity
7,165 Commits 878 Branches 229 Tags 2 GiB
f13f1f8fb8
Commit Graph

7165 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Masatoshi TSUCHIYA
ef102c4886
model_path should be ignored as the checkpoint path (#11157) 
* model_path is refered as the path of the trainer, and should be ignored as the checkpoint path.

* Improved according to Sgugger's comment.
 2021-04-12 09:06:41 -04:00
Sylvain Gugger
623cd6aef9 Fix style 2021-04-12 08:14:29 -04:00
cronoik
a99f7f5c75
Minor typos fixed (#11182) 2021-04-12 07:55:40 -04:00
Sylvain Gugger
26212c14e5 Reactivate Megatron tests an use less workers 2021-04-09 18:09:53 -04:00
Lysandre
716120cbd6 Fix Typo 2021-04-09 17:46:52 -04:00
Philipp Schmid
6f90c29eaa
added json dump and extraction of train run time (#11167) 
* added json dump and extraction of train run time

* make style happy
 2021-04-09 15:18:00 -04:00
Stas Bekman
07f0bb691d
[examples run_clm] fix _LazyModule hasher error (#11168) 
* fix _LazyModule hasher error

* reword
 2021-04-09 11:39:12 -07:00
Suraj Patil
c161dd56df
[examples/translation] support mBART-50 and M2M100 fine-tuning (#11170) 
* keep a list of multilingual tokenizers

* add forced_bos_token argument
 2021-04-09 23:58:42 +05:30
Kevin Canwen Xu
fb41f9f50c
Add a special tokenizer for CPM model (#11068) 
* Add a special tokenizer for CPM model

* make style

* fix

* Add docs

* styles

* cpm doc

* fix ci

* fix the overview

* add test

* make style

* typo

* Custom tokenizer flag

* Add REAMDE.md

Co-authored-by: Lysandre <lysandre.debut@reseau.eseo.fr>
 2021-04-10 02:07:47 +08:00
Sylvain Gugger
45fc8c7951
Make get_special_tokens_mask consider all tokens (#11163) 2021-04-09 11:57:44 -04:00
Saviour Owolabi
6060746570
Update README.md (#11161) 
Corrected a typo ('Downlowd' to 'Download')
 2021-04-09 11:52:21 -04:00
Keisuke Hirota
b9b60c1630
Fix LogitsProcessor documentation (#11130) 
* Change duplicated LogitsProcessor to LogitsWarper in LogitsProcessorList document

* Write more detailed information about LogitsProcessor's scores argument

* apply suggestion from review

* style

Co-authored-by: Suraj Patil <surajp815@gmail.com>
 2021-04-09 12:39:44 +05:30
Niklas Muennighoff
8b78a32be1
[Community notebooks] Add Wav2Vec notebook for creating captions for YT Clips (#11142) 
* Add Wav2Vec Inference notebook

* Update docs/source/community.md

Co-authored-by: Suraj Patil <surajp815@gmail.com>
 2021-04-09 12:10:37 +05:30
Stas Bekman
0311ba2153
typo (#11152) 
* typo

* style
 2021-04-08 19:47:31 -07:00
Sylvain Gugger
269c9638df Merge branch 'master' of github.com:huggingface/transformers 2021-04-08 21:14:56 -04:00
Sylvain Gugger
d31c7b104e Skip Megatron tests for now 2021-04-08 21:14:43 -04:00
Stas Bekman
c2e0fd5283
[setup] make fairscale and deepspeed setup extras (#11151) 
* make fairscale and deepspeed setup extras

* fix default

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* no reason not to ask for the good version

* update the CIs

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-04-08 15:46:54 -07:00
Sylvain Gugger
ba8b1f4754
Add support for multiple models for one config in auto classes (#11150) 
* Add support for multiple models for one config in auto classes

* Use get_values everywhere

* Prettier doc
 2021-04-08 18:41:36 -04:00
Stas Bekman
97ccf67bb3
[setup] extras[docs] must include 'all' (#11148) 
* extras[doc] must include 'all'

* fix

* better

* regroup
 2021-04-08 18:10:44 -04:00
Stas Bekman
66446909b2
[tests] relocate core integration tests (#11146) 
* relocate core integration tests

* add sys.path context manager

* cleanup

* try

* try2

* fix path

* doc

* style

* add dep

* add 2 more deps
 2021-04-08 13:13:17 -07:00
Andrea Cappelli
6c40e49712
Run mlm pad to multiple for fp16 (#11128) 
* Add mlm collator pad to multiple option (#10627)

* Use padding to 8x in run mlm (#10627)
 2021-04-08 16:12:49 -04:00
Sylvain Gugger
dfed4ec263
Don't duplicate logs in TensorBoard and handle --use_env (#11141) 2021-04-08 16:12:36 -04:00
Philipp Schmid
9c9b8e707b
Updates SageMaker docs for updating DLCs (#11140) 2021-04-08 16:05:53 -04:00
Lysandre Debut
ba2cf5f90d
Add fairscale and deepspeed back to the CI (#11147) 
* Add fairscale and deepspeed back to the CI

* Add deepspeed to single GPU tests
 2021-04-08 11:36:45 -07:00
Stas Bekman
1ed24afe91
[trainer] solve "scheduler before optimizer step" warning (#11144) 
* solve "scheduler before optimizer step" warning

* style

* correct the state evaluation test
 2021-04-08 11:28:48 -07:00
Julien Demouth
02ec02d6d3
Add nvidia megatron models (#10911) 
* Add support for NVIDIA Megatron models

* Add support for NVIDIA Megatron GPT2 and BERT

Add the megatron_gpt2 model. That model reuses the existing GPT2 model. This
commit includes a script to convert a Megatron-GPT2 checkpoint downloaded
from NVIDIA GPU Cloud. See examples/megatron-models/README.md for details.

Add the megatron_bert model. That model is implemented as a modification of
the existing BERT model in Transformers. This commit includes a script to
convert a Megatron-BERT checkpoint downloaded from NVIDIA GPU Cloud. See
examples/megatron-models/README.md for details.

* Update src/transformers/models/megatron_bert/configuration_megatron_bert.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Update src/transformers/models/megatron_bert/configuration_megatron_bert.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Update src/transformers/models/megatron_bert/configuration_megatron_bert.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Remove model.half in tests + add "# Copied ..."

Remove the model.half() instruction which makes tests fail on the CPU.

Add a comment "# Copied ..." before many classes in the model to enable automatic
tracking in CI between the new Megatron classes and the original Bert ones.

* Fix issues

* Fix Flax/TF tests

* Fix copyright

* Update src/transformers/models/megatron_bert/configuration_megatron_bert.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Update src/transformers/models/megatron_bert/configuration_megatron_bert.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Update src/transformers/models/megatron_bert/modeling_megatron_bert.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Update src/transformers/models/megatron_bert/modeling_megatron_bert.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Update src/transformers/models/megatron_bert/modeling_megatron_bert.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Update src/transformers/models/megatron_bert/modeling_megatron_bert.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Update docs/source/model_doc/megatron_bert.rst

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update docs/source/model_doc/megatron_gpt2.rst

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/megatron_bert/__init__.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/megatron_bert/modeling_megatron_bert.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/megatron_gpt2/convert_megatron_gpt2_checkpoint.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/megatron_gpt2/convert_megatron_gpt2_checkpoint.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/megatron_gpt2/convert_megatron_gpt2_checkpoint.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/megatron_bert/convert_megatron_bert_checkpoint.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/megatron_bert/convert_megatron_bert_checkpoint.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/megatron_bert/convert_megatron_bert_checkpoint.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/megatron_bert/modeling_megatron_bert.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/megatron_bert/modeling_megatron_bert.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/megatron_bert/modeling_megatron_bert.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/megatron_bert/modeling_megatron_bert.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/megatron_bert/modeling_megatron_bert.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/megatron_bert/modeling_megatron_bert.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/megatron_bert/modeling_megatron_bert.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/megatron_bert/modeling_megatron_bert.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/megatron_bert/modeling_megatron_bert.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/megatron_bert/modeling_megatron_bert.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/models/megatron_bert/modeling_megatron_bert.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Resolve most of 'sgugger' comments

* Fix conversion issue + Run make fix-copies/quality/docs

* Apply suggestions from code review

* Causal LM & merge

* Fix init

* Add CausalLM to last auto class

Co-authored-by: Julien Demouth <jdemouth@nvidia.com>
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Lysandre <lysandre.debut@reseau.eseo.fr>
 2021-04-08 14:09:11 -04:00
Stas Bekman
c6d664849b
[DeepSpeed] ZeRO Stage 3 (#10753) 
* synced gpus

* fix

* fix

* need to use t5-small for quality tests

* notes

* complete merge

* fix a disappearing std stream problem

* start zero3 tests

* wip

* tune params

* sorting out the pre-trained model loading

* reworking generate loop wip

* wip

* style

* fix tests

* split the tests

* refactor tests

* wip

* parameterized

* fix

* workout the resume from non-ds checkpoint pass + test

* cleanup

* remove no longer needed code

* split getter/setter functions

* complete the docs

* suggestions

* gpus and their compute capabilities link

* Apply suggestions from code review

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* style

* remove invalid paramgd

* automatically configure zero3 params that rely on hidden size

* make _get_resized_embeddings zero3-aware

* add test exercising resize_token_embeddings()

* add docstring

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
 2021-04-08 09:53:01 -07:00
Stas Bekman
acc851e1ff
[run_clm] clarify why we get the tokenizer warning on long input (#11145) 
* clarify why we get the warning here

* Update examples/language-modeling/run_clm.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* wording

* style

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-04-08 09:46:28 -07:00
Yusuke Mori
5bf5d50c8d
Typo fix of the name of BertLMHeadModel in BERT doc (#11133) 2021-04-08 08:22:58 -04:00
Jannis Born
f8e90d6fb9
Fix typing error in Trainer class (prediction_step) (#11138) 
* fix: docstrings in prediction_step

* ci: Satisfy line length requirements

* ci: character length requirements
 2021-04-08 08:22:25 -04:00
Sylvain Gugger
ffe0761777
Fix and refactor check_repo (#11127) 2021-04-07 17:56:21 -04:00
Philipp Schmid
3fd7eee18f
Adds use_auth_token with pipelines (#11123) 
* added model_kwargs to infer_framework_from_model

* added model_kwargs to tokenizer

* added use_auth_token as named parameter

* added dynamic get for use_auth_token
 2021-04-07 20:32:59 +02:00
Stas Bekman
1c15128312
[versions] handle version requirement ranges (#11110) 
* handle version requirement ranges

* add mixed requirement test

* cleanup
 2021-04-07 09:09:38 -07:00
Vasudev Gupta
7442801df5
fix tests (#11109) 2021-04-07 10:07:26 -04:00
Lysandre Debut
c0d97cee13
Adds a note to resize the token embedding matrix when adding special … (#11120) 
* Adds a note to resize the token embedding matrix when adding special tokens

* Remove superfluous space
 2021-04-07 10:06:45 -04:00
Sylvain Gugger
02f7c2fe66
Some styling of the training table in Notebooks (#11118) 2021-04-07 10:00:33 -04:00
Sylvain Gugger
11505fa139
Dummies multi backend (#11100) 
* Replaces requires_xxx by one generic method

* Quality and update check_dummies

* Fix inits check

* Post-merge cleanup
 2021-04-07 09:56:40 -04:00
Stas Bekman
424419f549
[examples] fix white space (#11099) 
these get concatenated without whitespace, so fix it
 2021-04-07 09:20:58 -04:00
Stas Bekman
c9035e4537
fix: The 'warn' method is deprecated (#11105) 
* The 'warn' method is deprecated

* fix test
 2021-04-07 09:20:06 -04:00
Leo Gao
247bed3857
GPTNeo: handle padded wte (#11079) 
* GPTNeo: handle padded wte

* Switch to config.vocab_size

* apply review suggestion

Co-authored-by: Suraj Patil <surajp815@gmail.com>
 2021-04-07 17:35:20 +05:30
cronoik
083ad7d46c
dead link fixed (#11103) 2021-04-07 07:50:47 -04:00
Sylvain Gugger
fd338abdeb Style 2021-04-06 19:54:13 -04:00
SHYAM SUNDER KUMAR
aef4cf8c52
accelerate question answering examples with no trainer (#11091) 
* accelerate question answering examples with no trainer

* removed train and eval flags also fixed fill np array function

* Update examples/question-answering/run_qa_beam_search_no_trainer.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update examples/question-answering/run_qa_no_trainer.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2021-04-06 19:35:21 -04:00
Sylvain Gugger
403d530eec
Auto feature extractor (#11097) 
* AutoFeatureExtractor

* Init and first tests

* Tests

* Damn you gitignore

* Quality

* Defensive test for when not all backends are here

* Use pattern for Speech2Text models
 2021-04-06 19:20:08 -04:00
Stas Bekman
520198f56f
[doc] gpt-neo (#11098) 
make the example work
 2021-04-06 16:42:06 -04:00
Lysandre
9853c5dd58 Development on v4.6.0dev0 2021-04-06 12:53:25 -04:00
Lysandre
4906a29f7f Release v4.5.0 2021-04-06 12:37:47 -04:00
Suraj Patil
2a8115f083
[WIP] GPT Neo cleanup (#10985) 
* better names

* add attention mixin

* all slow tests in one class

* make helper methods static so we can test

* add local attention tests

* better names

* doc

* apply review suggestions
 2021-04-06 12:24:15 -04:00
Philipp Schmid
76800fb8e6
added new merged Trainer test (#11090) 2021-04-06 15:12:21 +02:00
Philipp Schmid
b219d6b5a5
added social thumbnail for docs (#11083) 2021-04-06 14:56:18 +02:00