riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-08-03 03:31:05 +06:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity
13,067 Commits 892 Branches 232 Tags 2.1 GiB
7631db0fdc
Commit Graph

121 Commits

Author SHA1 Message Date
Yih-Dar
7631db0fdc
Pin deepspeed to 0.9.2 for now (#24024) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-06-05 20:00:28 +02:00
Yih-Dar
e69feab8a1
Update workflow files (#23658) 
* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-05-22 21:26:51 +02:00
Yih-Dar
aa4316757d
Change schedule CI time (#22884) 
* fix

* Update .github/workflows/self-nightly-past-ci-caller.yml

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2023-04-20 14:01:08 +02:00
Yih-Dar
648bd5a8aa
Show diff between 2 CI runs on Slack reports (#22798) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-04-19 19:27:37 +02:00
Yih-Dar
656d41ab4c
Remove DS_BUILD_AIO=1 (#22741) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-04-13 18:08:22 +02:00
Yih-Dar
0fe6c6bdca
(Re-)Enable Nightly + Past CI (#22393) 
* Enable Nightly + Past CI

* put schedule

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-03-30 21:06:35 +02:00
Yih-Dar
aab895c396
Make Slack CI reporting stronger (#21823) 
* Use token

* Avoid failure

* better error

* Fix

* fix style

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-02-28 17:12:44 +01:00
Yih-Dar
bf9a5882a7
Update some GH action versions (#20537) 
* update actions versions

* update actions versions

* update actions versions

* update actions versions

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-12-06 16:54:40 +01:00
Yih-Dar
67d32f4649
Replace set-output by $GITHUB_OUTPUT (#20547) 
* remove set-output

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-12-05 18:25:13 +01:00
Yih-Dar
e8d448edcf
extract warnings in GH workflows (#20487) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-11-29 15:58:54 +01:00
Yih-Dar
700e0cd65f
Add missing report button for Example test (#20293) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-11-17 15:55:00 +01:00
Yih-Dar
c06d555647
Show installed libraries and their versions in GA jobs (#20069) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-11-04 18:03:18 +01:00
Sylvain Gugger
9ac586b3c8
Rework pipeline tests (#19366) 
* Rework pipeline tests

* Try to fix Flax tests

* Try to put it before

* Use a new decorator instead

* Remove ignore marker since it doesn't work

* Filter pipeline tests

* Woopsie

* Use the fitlered list

* Clean up and fake modif

* Remove init

* Revert fake modif
 2022-10-07 18:01:58 -04:00
Yih-Dar
ba7f2173cc
Add runner availability check (#19054) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-09-19 12:27:06 +02:00
Yih-Dar
7a8118947f
Add checks for more workflow jobs (#18905) 
* add check for scheduled CI

* Add check to other CIs

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-09-07 12:51:37 +02:00
Yih-Dar
7d5fde991d
unpin slack_sdk version (#18901) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-09-06 18:42:00 +02:00
Yih-Dar
ecdf9b06bc
Remove cached torch_extensions on CI runners (#18868) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-09-02 18:17:58 +02:00
Yih-Dar
0ab465a5d2
pin Slack SDK to 3.18.1 to avoid failing issue (#18869) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-09-02 16:49:08 +02:00
Yih-Dar
d2704c4143
Add machine type in the artifact of Examples directory job (#18459) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-08-04 18:52:01 +02:00
Yih-Dar
ca169dbdf1
Enable PyTorch nightly build CI (#17335) 
* nightly build pytorch CI

* fix working dir

* change time and event name

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-06-17 16:42:27 +02:00
Yih-Dar
da0bed5f4a
Pre-build DeepSpeed (#17607) 
* pre-build deepspeed

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-06-09 23:02:33 +02:00
Yih-Dar
9aa230aa2f
Use latest stable PyTorch/DeepSpeed for Push & Scheduled CI (#17417) 
* update versions

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-06-07 11:53:05 +02:00
Yih-Dar
659b27fd26
Print more library versions in CI (#17384) 
* print more lib. versions and just befor test runs

* update print_env_pt.py

* rename to print_env

* Disable warning + better job name

* print python version

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-06-02 10:24:16 +02:00
Yih-Dar
1ac2b8fa7f
Fix missing job action button in CI report (#17270) 
* use matrix.machine_type

* fix job names used in job_link

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-05-17 08:31:06 +02:00
Yih-Dar
506899d147
Fix push CI channel (#17242) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-05-13 20:59:56 +02:00
Yih-Dar
993553b2f1
fix --gpus option for docker (#17235) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-05-13 17:26:26 +02:00
Yih-Dar
38043d8453
Update self-push workflow (#17177) 
* update push ci

* install git-python

* update comment

* update deepspeed jobs

* fix report

* skip 2 more tests that require fairscale

* Fix changes in test_fetcher.py (to deal with `setup.py` is changed)

* set RUN_PT_TF_CROSS_TESTS=1 and final clean-up

* remove SIGOPT_API_TOKEN

* remove echo "$matrix_folders"

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-05-13 16:28:00 +02:00
Yih-Dar
3212afa614
split single_gpu and multi_gpu (#17083) 
* split single_gpu and multi_gpu

* update needs in send_result

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-05-09 07:13:07 -04:00
Yih-Dar
19420fd99e
Move test model folders (#17034) 
* move test model folders (TODO: fix imports and others)

* fix (potentially partially) imports (in model test modules)

* fix (potentially partially) imports (in tokenization test modules)

* fix (potentially partially) imports (in feature extraction test modules)

* fix import utils.test_modeling_tf_core

* fix path ../fixtures/

* fix imports about generation.test_generation_flax_utils

* fix more imports

* fix fixture path

* fix get_test_dir

* update module_to_test_file

* fix get_tests_dir from wrong transformers.utils

* update config.yml (CircleCI)

* fix style

* remove missing imports

* update new model script

* update check_repo

* update SPECIAL_MODULE_TO_TEST_MAP

* fix style

* add __init__

* update self-scheduled

* fix add_new_model scripts

* check one way to get location back

* python setup.py build install

* fix import in test auto

* update self-scheduled.yml

* update slack notification script

* Add comments about artifact names

* fix for yolos

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-05-03 14:42:02 +02:00
Stas Bekman
3a16ab25c8
[self-scheduled ci] explain where dependencies are (#16757) 2022-04-13 12:28:02 -04:00
Yih-Dar
09013efdf1
Update step name (#16189) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-03-16 11:19:38 -04:00
Lysandre Debut
a01fe4cd32
Rebuild deepspeed (#16081) 
* Rebuild deepspeed

* Apply suggestions from code review

Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>

Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
 2022-03-11 14:35:48 -05:00
Lysandre Debut
54f0db4066
Add PT + TF automatic builds (#15860) 
* Add PT + TF automatic builds

* Apply suggestions from code review

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Wrap up

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2022-03-01 08:55:11 -05:00
Lysandre
c475f3ce2d Scheduled tests should only run on a daily basis 2022-02-23 17:52:22 -05:00
Lysandre Debut
4c737f0e40
[Test refactor 4/5] Improve the scheduled tests (#15728) 2022-02-23 15:48:05 -05:00
Lysandre Debut
29c10a41d0
[Test refactor 1/5] Per-folder tests reorganization (#15725) 
* Per-folder tests reorganization

Co-authored-by: sgugger <sylvain.gugger@gmail.com>
Co-authored-by: Stas Bekman <stas@stason.org>
 2022-02-23 15:46:28 -05:00
Patrick von Platen
e2b6e73fa2
[Flax tests] Disable scheduled GPU tests (#15503) 2022-02-03 17:12:14 +01:00
Ayush Chaurasia
c74f3d4c48
Add W&B backend for hyperparameter sweep (#14582) 
# Add support for W&B hyperparameter sweep
This PR:
* allows using wandb for running hyperparameter search.
* The runs are visualized on W&B sweeps dashboard
* This supports runnning sweeps on parallel devices, all reporting to the same central dashboard.

### Usage
**To run new a hyperparameter search:**
```
trainer.hyperparameter_search(
    backend="wandb", 
    project="transformers_sweep", # name of the project
    n_trials=5,
    metric="eval/loss", # metric to be optimized, default 'eval/loss'. A warning is raised if the passed metric is not found
)
```
This outputs a sweep id. Eg. `my_project/sweep_id`

**To run sweeps on parallel devices:**
Just pass sweep id which you want to run parallel
```
trainer.hyperparameter_search(
    backend="wandb", 
    sweep_id = "my_project/sweep_id"
)
```
 2022-02-02 14:06:14 -05:00
Lysandre Debut
444ea95a80
Print out durations of all scheduled tests (#15102) 2022-01-11 08:15:59 -05:00
NielsRogge
83c552d390
Add detectron2 to Github actions (#15053) 2022-01-06 08:53:58 -05:00
Patrick von Platen
dbac8899fe
[Tests] Correct Wav2Vec2 & WavLM tests (#15015) 
* up

* up

* up
 2022-01-03 20:19:04 +01:00
Patrick von Platen
c4a96cecbc
Wav2Vec2 meets phonemes (#14353) 
* up

* add tokenizer

* improve more

* finish tokenizer

* finish

* adapt speech recognition script

* adapt convert

* more fixes

* more fixes

* update phonemizer wav2vec2

* better naming

* fix more tests

* more fixes swedish

* correct tests

* finish

* improve script

* remove file

* up

* lets get those 100 model architectures until the end of the month

* make fix-copies

* correct more

* correct script

* more fixes

* more fixes

* add to docs

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* replace assert

* fix copies

* fix docs

* new try docs

* boom boom

* update

* add phonemizer to audio tests

* make fix-copies

* up

* upload models

* some changes

* Update tests/test_tokenization_wav2vec2_phoneme.py

Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>

* more fixes

* remove @

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
 2021-12-17 19:56:44 +01:00
Patrick von Platen
961732c276
[Wav2Vec2] PyCTCDecode Integration to support language model boosted decoding (#14339) 
* up

* up

* up

* make it cleaner

* correct

* make styhahalal

* add more tests

* finish

* small fix

* make style

* up

* tryout to solve cicrle ci

* up

* fix more tests

* fix more tests

* apply sylvains suggestions

* fix import

* correct docs

* add pyctcdecode only to speech tests

* fix more tests

* add tf, flax and pt tests

* add pt

* fix last tests

* fix more tests

* Apply suggestions from code review

* change lines

* Apply suggestions from code review

Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>

* correct tests

* correct tests

* add doc string

Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
 2021-12-08 12:07:54 +01:00
Stas Bekman
e62091d5a7
[CI] move env print to util, add pt, nccl versions (#14607) 
* move env print to util, add pt, nccl versions

* style

* version

* align
 2021-12-03 08:18:36 -05:00
Stas Bekman
d1fd64e7aa
clear ~/.cache/torch_extensions between builds (#14520) 2021-11-25 03:15:35 -05:00
NielsRogge
3772af49ce
[Tests] Improve vision tests (#14458) 
* Improve tests

* Install vision for tf tests
 2021-11-24 15:22:20 +01:00
Patrick von Platen
01b1466983
[TPU tests] Enable first TPU examples pytorch (#14121) 
* up

* up

* fix

* up

* Update examples/pytorch/test_xla_examples.py

* correct labels

* up

* up

* up

* up

* up

* up
 2021-10-28 01:22:28 +02:00
kding1
6a3a197fcd
Add SigOpt HPO to transformers trainer api (#13572) 
* add sigopt hpo to transformers.

Signed-off-by: Ding, Ke <ke.ding@intel.com>

* extend sigopt changes to test code and others..

Signed-off-by: Ding, Ke <ke.ding@intel.com>

* Style.

* fix style for sigopt integration.

Signed-off-by: Ding, Ke <ke.ding@intel.com>

* Add necessary information to run unittests on SigOpt.

Co-authored-by: Morgan Funtowicz <funtowiczmo@gmail.com>
 2021-09-23 17:01:51 +02:00
Anton Lozhkov
f8363e49f9
Install libsndfile (#13403) 2021-09-06 17:12:43 -04:00
Patrick von Platen
642e1936e3
[GitHub Runner] Fix flax runner (#13357) 
* correct

* also comment out multi-gpu test push
 2021-08-31 09:01:35 -04:00