riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-31 02:02:21 +06:00
Code Issues Actions 23 Packages Projects Releases Wiki Activity
6,840 Commits 887 Branches 232 Tags 2.1 GiB
1f5ea9e04a
Commit Graph

6840 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Eliza Szczechla
1f5ea9e04a
Add notebook on fine-tuning Bart (#10883) 
Co-authored-by: Eliza <eliza@habanero.tiger.com.pl>
 2021-03-24 11:03:37 -04:00
imzhengzx
f81077fcf3
error type of tokenizer in __init__ definition (#10879) 
the orignal code in line 246 is
```
tokenizer: Optional["PreTrainedTokenizerBase"] = None,
```

it should be
```
tokenizer: Optional[PreTrainedTokenizerBase] = None,
```
 2021-03-24 11:00:14 -04:00
Sylvain Gugger
1aed2b908e
Add new notebook links in the docs (#10876) 2021-03-24 09:45:08 -04:00
Sylvain Gugger
a735f727cc
Fix test_trainer_distributed (#10875) 2021-03-23 19:03:06 -04:00
Philipp Schmid
8c297cdb30
Sm trainer smp init fix (#10870) 
* rewrote is_sagemaker_model_parallel_available

* added is_sagemaker_model_parallel_available to SageMakerTrainer

* removed unnecessary mp_parameters as TrainingArguments

* make style happy

* added mp_parameters again to parse mp-specific args.
 2021-03-23 20:07:55 +01:00
RafaelWO
d4d4447d53
fixed prefix_allowed_tokens_fn docstring in generate() (#10862) 2021-03-23 13:48:22 -04:00
Bhadresh Savani
7ef40120a0
[Examples] Added predict stage and Updated Example Template (#10868) 
* added predict stage

* added test keyword in exception message

* removed example specific saving predictions

* fixed f-string error

* removed extra line

Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>

Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
 2021-03-23 10:37:59 -07:00
Stas Bekman
fb2b89840b
[file_utils] import refactor (#10859) 
* import refactor

* fix the fallback
 2021-03-23 09:41:41 -07:00
Lysandre
3f48b2bc3e Update stable docs 2021-03-23 11:01:16 -04:00
Philipp Schmid
77ffd5edd5
Amazon SageMaker Documentation (#10867) 
* added finished documentation

* changed version from 1.6 to 1.6.0 for distributed

* updated versions

* updated urls
 2021-03-23 10:56:44 -04:00
Sylvain Gugger
bf1f43fbd7
Update the example template for a no Trainer option (#10865) 2021-03-23 10:02:39 -04:00
Marta Maślankowska
2eb596f085
Fix p_mask cls token masking in qa pipeline (#10863) 2021-03-23 09:08:39 -04:00
Bhadresh Savani
eb330e8904
fixed typo (#10861) 2021-03-23 08:15:28 -04:00
Stas Bekman
e21f89f64c
fix nan in full-fp16 label_smoothing eval (#10815) 2021-03-22 19:23:24 -07:00
Sylvain Gugger
b5b957a65c
Make convert_to_onnx runable as script again (#10857) 2021-03-22 22:16:39 -04:00
Patrick von Platen
77bf3fe787
[Generate] Add save mode logits processor to remove nans and infs if necessary (#10769) 
* push

* finish

* finish

* make fix copies

* change name
 2021-03-23 01:00:05 +03:00
Eliza Szczechla
9f8fa4e973
Use DataCollatorForSeq2Seq in run_summarization in all cases (#10856) 
Co-authored-by: Eliza <eliza@habanero.tiger.com.pl>
 2021-03-22 15:05:39 -04:00
Ruan Chaves
a8d4d6776d
Modify the Trainer class to handle simultaneous execution of Ray Tune and Weights & Biases (#10823) 
* Modify the _hp_search_setup method on the Trainer class to handle the wandb argument passed by Ray Tune to model config.

* Reformat single quotes as double quotes.
 2021-03-22 14:04:51 -04:00
Boris Dayma
125ccead71
feat(wandb): logging and configuration improvements (#10826) 
* feat: ensure unique artifact id

* feat: allow manual init

* fix: simplify reinit logic

* fix: no dropped value + immediate commits

* fix: wandb use in sagemaker

* docs: improve documenation and formatting

* fix: typos

* docs: improve formatting
 2021-03-22 10:45:17 -04:00
Sidd Karamcheti
b230181d41
Add simple one character fix so that on_step_begin and on_step_end are called at the right times (#10839) 2021-03-22 09:15:39 -04:00
Stas Bekman
24ab5b08a3
[makefile] autogenerate target (#10814) 
* autogenerate target

* clarify comment
 2021-03-22 09:14:22 -04:00
Sebastian Olsson
2c6684239f
Correct AutoConfig call docstrings (#10822) 2021-03-22 09:12:44 -04:00
Stas Bekman
8fb4671811
[vulnerability] in example deps fix (#10817) 
Takes care of:
https://github.com/huggingface/transformers/security/dependabot/examples/research_projects/lxmert/requirements.txt/jinja2/open

@LysandreJik

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
 2021-03-22 09:05:24 -04:00
dependabot[bot]
dbfe379514
Bump jinja2 from 2.11.2 to 2.11.3 in /examples/research_projects/lxmert (#10818) 
Bumps [jinja2](https://github.com/pallets/jinja) from 2.11.2 to 2.11.3.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/master/CHANGES.rst)
- [Commits](https://github.com/pallets/jinja/compare/2.11.2...2.11.3)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-03-22 08:54:50 -04:00
Qiushi Pan
29904a967b
Update FINE_TUNE_XLSR_WAV2VEC2.md (#10849) 
Fix typo.
 2021-03-22 07:58:59 -04:00
Patrick von Platen
0f226f78ce
push (#10846) 2021-03-22 10:32:21 +03:00
Suraj Patil
82b8d8c7b0
Update FINE_TUNE_XLSR_WAV2VEC2.md 2021-03-21 22:47:09 +05:30
Patrick von Platen
af6125ffdb
Update FINE_TUNE_XLSR_WAV2VEC2.md 2021-03-21 12:31:33 +03:00
Patrick von Platen
5aaf6e1460
small improvements for wav2vec2 info script (#10829) 2021-03-21 11:41:44 +03:00
Eric Lam
be87b84276
Add new community notebook - wav2vec2 with GPT (#10794) 
* Add new community notebook - wav2vec2 with GPT

* Update:community.md, new nb add
* feat: notebook of wav2vec xlsr ctc decoding with gpt logit adjustment
* Update: Wav2vec2 CTC decoding with gpt2 adjustment

* Update docs/source/community.md

Co-authored-by: Suraj Patil <surajp815@gmail.com>
 2021-03-21 13:29:53 +05:30
Suraj Patil
68b55885ed
add doc for Local machine (#10828) 2021-03-21 13:25:34 +05:30
Sylvain Gugger
21e86f99e6
Sort init import (#10801) 
* Initial script

* Add script to properly sort imports in init.

* Add to the CI

* Update utils/custom_init_isort.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Separate scripts that change content from quality

* Move class_mapping_update to style_checks

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
 2021-03-19 16:17:13 -04:00
Julien Chaumond
1438c487df
wav2vec doc tweaks (#10808) 
* wording/typos tweaks

* Make model upload instructions simpler
 2021-03-19 12:48:54 -04:00
Patrick von Platen
b9570a813c
Update FINE_TUNE_XLSR_WAV2VEC2.md 2021-03-19 19:45:28 +03:00
Philipp Schmid
f2b744f690
Add transformers id to hub requests (#10811) 
* add uuid.hext to user_agent

* add log

* changed order of it

* renamed as session id

* renamed variable

* reverted naming of the const
 2021-03-19 16:26:32 +01:00
Sylvain Gugger
946400fb68
Expand a bit the presentation of examples (#10799) 
* Expand a bit the presentation of examples

* Apply suggestions from code review

Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>

* Address review comments

Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
 2021-03-19 10:06:08 -04:00
Bhadresh Savani
fd1d9f1ab8
[Example] Updating Question Answering examples for Predict Stage (#10792) 
* added prediction stage and eval fix

* style correction

* removed extra lines
 2021-03-19 09:42:17 -04:00
Patrick von Platen
e8968bd03a
[XLSR-Wav2Vec2 Info doc] Add a couple of lines (#10806) 
* finish

* fix

* fix

* fix

* fix
 2021-03-19 12:52:54 +03:00
Théo Matussière
117dba9948
fix backend tokenizer args override: key mismatch (#10686) 
* fix backend tokenizer args override: key mismatch

* no touching the docs

* fix mpnet

* add mpnet to test

* fix test

Co-authored-by: theo <theo@matussie.re>
 2021-03-18 22:13:45 -04:00
Stas Bekman
427ea3fecb
addressing vulnerability report in research project deps (#10802) 
Following up on a security alert:
https://github.com/huggingface/transformers/security/dependabot/examples/research_projects/lxmert/requirements.txt/Pillow/open
 2021-03-18 22:02:10 -04:00
Patrick von Platen
2ae678229f
Update FINE_TUNE_XLSR_WAV2VEC2.md 2021-03-19 00:29:20 +03:00
Patrick von Platen
68a3215949
Update FINE_TUNE_XLSR_WAV2VEC2.md 2021-03-19 00:27:40 +03:00
Patrick von Platen
03df3fbcb4
Update FINE_TUNE_XLSR_WAV2VEC2.md 2021-03-19 00:26:49 +03:00
Patrick von Platen
e84adbed40
Add XLSR-Wav2Vec2 Fine-Tuning README.md (#10786) 
* upload

* upload fine-tuning script

* improve

* adapt

* Apply suggestions from code review

* correct

* upload

* finalize

* remove @

* correct typos
 2021-03-19 00:22:43 +03:00
Sylvain Gugger
dcebe254fa Document v4.4.2 2021-03-18 15:19:25 -04:00
Sylvain Gugger
008672e6e5
Fix distributed evaluation (#10795) 
* Fix distributed evaluation

* Use logger
 2021-03-18 13:12:04 -04:00
Stas Bekman
9352b5151a
[examples/seq2seq/README.md] fix t5 examples (#10734) 
* [examples/seq2seq] fix t5 examples

This PR:
* fixes T5 examples to include `--source_prefix` - it's **not** optional. If you give it a try you will see that you get 10x worse bleu scores w/o it. w/ `27.6849`, w/ `2.374`
* added a normal translation example w/o the peculiarities of MBart and T5
* reduces the default max samples to 50 so it's much faster to test quickly

summarization seems to be broken for t5 score-wise: https://github.com/huggingface/transformers/issues/10733

@sgugger

* specify explicitly the t5 models requiring the special handling

* one more

* update the t5 summarization example to use cnn_dailymail

* move max*samples into the top level README.md

* better wording

* better wording
 2021-03-18 09:55:39 -07:00
Vimarsh Chaturvedi
094afa515d
from_pretrained: check that the pretrained model is for the right model architecture (#10586) 
* Added check to ensure model name passed to from_pretrained and model are the same

* Added test to check from_pretrained throws assert error when passed an incompatiable model name

* Modified assert in from_pretrained with f-strings. Modified test to ensure desired assert message is being generated

* Added check to ensure config and model has model_type

* Fix FlauBERT heads

Co-authored-by: vimarsh chaturvedi <vimarsh chaturvedi>
Co-authored-by: Stas Bekman <stas@stason.org>
Co-authored-by: Lysandre <lysandre.debut@reseau.eseo.fr>
 2021-03-18 12:51:42 -04:00
Julien Chaumond
4f3e93cfaf
[file_utils] do not gobble certain kinds of requests.ConnectionError (#10235) 
* do not gobble certain kinds of requests.ConnectionError

* Apply review comments

Co-authored-by: Lysandre <lysandre.debut@reseau.eseo.fr>
 2021-03-18 12:37:45 -04:00
James Thomin
ce9724e1bd
Fix bug in input check for LengthGroupSampler (#10783) 
This commit fixes a bug in the LengthGroupSampler where if
model_input_name is not set, the default value is None instead of
"input_ids"
 2021-03-18 10:25:57 -04:00