riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-08-03 03:31:05 +06:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity
11,523 Commits 892 Branches 232 Tags 2.1 GiB
bf9a5882a7
Commit Graph

114 Commits

Author SHA1 Message Date
Yih-Dar
bf9a5882a7
Update some GH action versions (#20537) 
* update actions versions

* update actions versions

* update actions versions

* update actions versions

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-12-06 16:54:40 +01:00
Yih-Dar
67d32f4649
Replace set-output by $GITHUB_OUTPUT (#20547) 
* remove set-output

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-12-05 18:25:13 +01:00
Yih-Dar
e8d448edcf
extract warnings in GH workflows (#20487) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-11-29 15:58:54 +01:00
Yih-Dar
700e0cd65f
Add missing report button for Example test (#20293) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-11-17 15:55:00 +01:00
Yih-Dar
c06d555647
Show installed libraries and their versions in GA jobs (#20069) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-11-04 18:03:18 +01:00
Sylvain Gugger
9ac586b3c8
Rework pipeline tests (#19366) 
* Rework pipeline tests

* Try to fix Flax tests

* Try to put it before

* Use a new decorator instead

* Remove ignore marker since it doesn't work

* Filter pipeline tests

* Woopsie

* Use the fitlered list

* Clean up and fake modif

* Remove init

* Revert fake modif
 2022-10-07 18:01:58 -04:00
Yih-Dar
ba7f2173cc
Add runner availability check (#19054) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-09-19 12:27:06 +02:00
Yih-Dar
7a8118947f
Add checks for more workflow jobs (#18905) 
* add check for scheduled CI

* Add check to other CIs

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-09-07 12:51:37 +02:00
Yih-Dar
7d5fde991d
unpin slack_sdk version (#18901) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-09-06 18:42:00 +02:00
Yih-Dar
ecdf9b06bc
Remove cached torch_extensions on CI runners (#18868) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-09-02 18:17:58 +02:00
Yih-Dar
0ab465a5d2
pin Slack SDK to 3.18.1 to avoid failing issue (#18869) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-09-02 16:49:08 +02:00
Yih-Dar
d2704c4143
Add machine type in the artifact of Examples directory job (#18459) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-08-04 18:52:01 +02:00
Yih-Dar
ca169dbdf1
Enable PyTorch nightly build CI (#17335) 
* nightly build pytorch CI

* fix working dir

* change time and event name

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-06-17 16:42:27 +02:00
Yih-Dar
da0bed5f4a
Pre-build DeepSpeed (#17607) 
* pre-build deepspeed

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-06-09 23:02:33 +02:00
Yih-Dar
9aa230aa2f
Use latest stable PyTorch/DeepSpeed for Push & Scheduled CI (#17417) 
* update versions

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-06-07 11:53:05 +02:00
Yih-Dar
659b27fd26
Print more library versions in CI (#17384) 
* print more lib. versions and just befor test runs

* update print_env_pt.py

* rename to print_env

* Disable warning + better job name

* print python version

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-06-02 10:24:16 +02:00
Yih-Dar
1ac2b8fa7f
Fix missing job action button in CI report (#17270) 
* use matrix.machine_type

* fix job names used in job_link

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-05-17 08:31:06 +02:00
Yih-Dar
506899d147
Fix push CI channel (#17242) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-05-13 20:59:56 +02:00
Yih-Dar
993553b2f1
fix --gpus option for docker (#17235) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-05-13 17:26:26 +02:00
Yih-Dar
38043d8453
Update self-push workflow (#17177) 
* update push ci

* install git-python

* update comment

* update deepspeed jobs

* fix report

* skip 2 more tests that require fairscale

* Fix changes in test_fetcher.py (to deal with `setup.py` is changed)

* set RUN_PT_TF_CROSS_TESTS=1 and final clean-up

* remove SIGOPT_API_TOKEN

* remove echo "$matrix_folders"

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-05-13 16:28:00 +02:00
Yih-Dar
3212afa614
split single_gpu and multi_gpu (#17083) 
* split single_gpu and multi_gpu

* update needs in send_result

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-05-09 07:13:07 -04:00
Yih-Dar
19420fd99e
Move test model folders (#17034) 
* move test model folders (TODO: fix imports and others)

* fix (potentially partially) imports (in model test modules)

* fix (potentially partially) imports (in tokenization test modules)

* fix (potentially partially) imports (in feature extraction test modules)

* fix import utils.test_modeling_tf_core

* fix path ../fixtures/

* fix imports about generation.test_generation_flax_utils

* fix more imports

* fix fixture path

* fix get_test_dir

* update module_to_test_file

* fix get_tests_dir from wrong transformers.utils

* update config.yml (CircleCI)

* fix style

* remove missing imports

* update new model script

* update check_repo

* update SPECIAL_MODULE_TO_TEST_MAP

* fix style

* add __init__

* update self-scheduled

* fix add_new_model scripts

* check one way to get location back

* python setup.py build install

* fix import in test auto

* update self-scheduled.yml

* update slack notification script

* Add comments about artifact names

* fix for yolos

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-05-03 14:42:02 +02:00
Stas Bekman
3a16ab25c8
[self-scheduled ci] explain where dependencies are (#16757) 2022-04-13 12:28:02 -04:00
Yih-Dar
09013efdf1
Update step name (#16189) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-03-16 11:19:38 -04:00
Lysandre Debut
a01fe4cd32
Rebuild deepspeed (#16081) 
* Rebuild deepspeed

* Apply suggestions from code review

Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>

Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
 2022-03-11 14:35:48 -05:00
Lysandre Debut
54f0db4066
Add PT + TF automatic builds (#15860) 
* Add PT + TF automatic builds

* Apply suggestions from code review

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Wrap up

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2022-03-01 08:55:11 -05:00
Lysandre
c475f3ce2d Scheduled tests should only run on a daily basis 2022-02-23 17:52:22 -05:00
Lysandre Debut
4c737f0e40
[Test refactor 4/5] Improve the scheduled tests (#15728) 2022-02-23 15:48:05 -05:00
Lysandre Debut
29c10a41d0
[Test refactor 1/5] Per-folder tests reorganization (#15725) 
* Per-folder tests reorganization

Co-authored-by: sgugger <sylvain.gugger@gmail.com>
Co-authored-by: Stas Bekman <stas@stason.org>
 2022-02-23 15:46:28 -05:00
Patrick von Platen
e2b6e73fa2
[Flax tests] Disable scheduled GPU tests (#15503) 2022-02-03 17:12:14 +01:00
Ayush Chaurasia
c74f3d4c48
Add W&B backend for hyperparameter sweep (#14582) 
# Add support for W&B hyperparameter sweep
This PR:
* allows using wandb for running hyperparameter search.
* The runs are visualized on W&B sweeps dashboard
* This supports runnning sweeps on parallel devices, all reporting to the same central dashboard.

### Usage
**To run new a hyperparameter search:**
```
trainer.hyperparameter_search(
    backend="wandb", 
    project="transformers_sweep", # name of the project
    n_trials=5,
    metric="eval/loss", # metric to be optimized, default 'eval/loss'. A warning is raised if the passed metric is not found
)
```
This outputs a sweep id. Eg. `my_project/sweep_id`

**To run sweeps on parallel devices:**
Just pass sweep id which you want to run parallel
```
trainer.hyperparameter_search(
    backend="wandb", 
    sweep_id = "my_project/sweep_id"
)
```
 2022-02-02 14:06:14 -05:00
Lysandre Debut
444ea95a80
Print out durations of all scheduled tests (#15102) 2022-01-11 08:15:59 -05:00
NielsRogge
83c552d390
Add detectron2 to Github actions (#15053) 2022-01-06 08:53:58 -05:00
Patrick von Platen
dbac8899fe
[Tests] Correct Wav2Vec2 & WavLM tests (#15015) 
* up

* up

* up
 2022-01-03 20:19:04 +01:00
Patrick von Platen
c4a96cecbc
Wav2Vec2 meets phonemes (#14353) 
* up

* add tokenizer

* improve more

* finish tokenizer

* finish

* adapt speech recognition script

* adapt convert

* more fixes

* more fixes

* update phonemizer wav2vec2

* better naming

* fix more tests

* more fixes swedish

* correct tests

* finish

* improve script

* remove file

* up

* lets get those 100 model architectures until the end of the month

* make fix-copies

* correct more

* correct script

* more fixes

* more fixes

* add to docs

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* replace assert

* fix copies

* fix docs

* new try docs

* boom boom

* update

* add phonemizer to audio tests

* make fix-copies

* up

* upload models

* some changes

* Update tests/test_tokenization_wav2vec2_phoneme.py

Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>

* more fixes

* remove @

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
 2021-12-17 19:56:44 +01:00
Patrick von Platen
961732c276
[Wav2Vec2] PyCTCDecode Integration to support language model boosted decoding (#14339) 
* up

* up

* up

* make it cleaner

* correct

* make styhahalal

* add more tests

* finish

* small fix

* make style

* up

* tryout to solve cicrle ci

* up

* fix more tests

* fix more tests

* apply sylvains suggestions

* fix import

* correct docs

* add pyctcdecode only to speech tests

* fix more tests

* add tf, flax and pt tests

* add pt

* fix last tests

* fix more tests

* Apply suggestions from code review

* change lines

* Apply suggestions from code review

Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>

* correct tests

* correct tests

* add doc string

Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
 2021-12-08 12:07:54 +01:00
Stas Bekman
e62091d5a7
[CI] move env print to util, add pt, nccl versions (#14607) 
* move env print to util, add pt, nccl versions

* style

* version

* align
 2021-12-03 08:18:36 -05:00
Stas Bekman
d1fd64e7aa
clear ~/.cache/torch_extensions between builds (#14520) 2021-11-25 03:15:35 -05:00
NielsRogge
3772af49ce
[Tests] Improve vision tests (#14458) 
* Improve tests

* Install vision for tf tests
 2021-11-24 15:22:20 +01:00
Patrick von Platen
01b1466983
[TPU tests] Enable first TPU examples pytorch (#14121) 
* up

* up

* fix

* up

* Update examples/pytorch/test_xla_examples.py

* correct labels

* up

* up

* up

* up

* up

* up
 2021-10-28 01:22:28 +02:00
kding1
6a3a197fcd
Add SigOpt HPO to transformers trainer api (#13572) 
* add sigopt hpo to transformers.

Signed-off-by: Ding, Ke <ke.ding@intel.com>

* extend sigopt changes to test code and others..

Signed-off-by: Ding, Ke <ke.ding@intel.com>

* Style.

* fix style for sigopt integration.

Signed-off-by: Ding, Ke <ke.ding@intel.com>

* Add necessary information to run unittests on SigOpt.

Co-authored-by: Morgan Funtowicz <funtowiczmo@gmail.com>
 2021-09-23 17:01:51 +02:00
Anton Lozhkov
f8363e49f9
Install libsndfile (#13403) 2021-09-06 17:12:43 -04:00
Patrick von Platen
642e1936e3
[GitHub Runner] Fix flax runner (#13357) 
* correct

* also comment out multi-gpu test push
 2021-08-31 09:01:35 -04:00
Lysandre
aa08a34669 [Flax tests] NVIDIA-SMI failure should continue 2021-08-31 14:18:20 +02:00
Patrick von Platen
062300ba7f
[Testing] Add Flax Tests on GPU, Add Speech and Vision to Flax & TF tests (#13313) 
* up

* finish

* Apply suggestions from code review

* apply Lysandres suggestions

* adapt circle ci as well

* finish

* Update setup.py
 2021-08-31 11:08:22 +02:00
Lysandre Debut
3bb4466260
Better notification service (#13267) 2021-08-25 12:14:44 -04:00
Lysandre Debut
e2f07c01e9
Ci continue through smi failure (#13140) 
* Continue on error

* Specific

* Temporary patch
 2021-08-16 11:40:38 -04:00
Lysandre Debut
c89180a9de
Install git (#13091) 
* Install git

* Add TF tests

* And last TF test

* Add in commented code too

Co-authored-by: Sylvain Gugger <sylvain.gugger@gmail.com>
 2021-08-11 18:09:41 +02:00
Lysandre Debut
546dc24e08
Longer timeout for slow tests (#12779) 2021-07-19 04:55:40 -04:00
Lysandre Debut
83f025125d
Add timeout to CI. (#12684) 
* Global 60-300 seconds timeout

* Add verbose option

* [skip ci] typo
 2021-07-13 15:13:18 -04:00