Marc Sun
bb76f81e40
[CI] Quantization workflow fix ( #30158 )
...
* fix workflow
* call ci
* Update .github/workflows/self-scheduled-caller.yml
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
---------
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
2024-04-10 11:51:06 +02:00
Marc Sun
6cdbd73e01
[CI] Fix setup ( #30147 )
...
* [CI] fix setup
* fix
* test
* Revert "test"
This reverts commit 7df416d450
2024-04-09 18:10:00 +02:00
Marc Sun
58a939c6b7
Fix quantization tests ( #29914 )
...
* revert back to torch 2.1.1
* run test
* switch to torch 2.2.1
* udapte dockerfile
* fix awq tests
* fix test
* run quanto tests
* update tests
* split quantization tests
* fix
* fix again
* final fix
* fix report artifact
* build docker again
* Revert "build docker again"
This reverts commit 399a5f9d93
2024-04-09 17:10:29 +02:00
Yih-Dar
b17b54d3dd
Refactor daily CI workflow ( #30012 )
...
* separate jobs
* separate jobs
* use channel name directly instead of ID
* use channel name directly instead of ID
* use channel name directly instead of ID
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2024-04-05 15:49:51 +02:00
Marc Sun
f54d82cace
[CI] Quantization workflow ( #29046 )
...
* [CI] Quantization workflow
* build dockerfile
* fix dockerfile
* update self-cheduled.yml
* test build dockerfile on push
* fix torch install
* udapte to python 3.10
* update aqlm version
* uncomment build dockerfile
* tests if the scheduler works
* fix docker
* do not trigger on psuh again
* add additional runs
* test again
* all good
* style
* Update .github/workflows/self-scheduled.yml
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
* test build dockerfile with torch 2.2.0
* fix extra
* clean
* revert changes
* Revert "revert changes"
This reverts commit 4cb52b8822
2024-02-28 10:09:25 -05:00
Yih-Dar
93f8617afd
Use DS_DISABLE_NINJA=1 ( #29290 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2024-02-26 17:41:01 +08:00
Yih-Dar
4735866141
Split daily CI using 2 level matrix ( #28773 )
...
* update / add new workflow files
* Add comment
* Use env.NUM_SLICES
* use scripts
* use scripts
* use scripts
* Fix
* using one script
* Fix
* remove unused file
* update
* fail-fast: false
* remove unused file
* fix
* fix
* use matrix
* inputs
* style
* update
* fix
* fix
* no model name
* add doc
* allow args
* style
* pass argument
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2024-01-31 18:04:43 +01:00
Yih-Dar
95346e9dcd
Add artifact name in job step to maintain job / artifact correspondence ( #28682 )
...
* avoid using job name
* apply to other files
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2024-01-31 15:58:17 +01:00
Patrick von Platen
cbbe30749b
[Whisper] Fix slow test ( #28407 )
...
* [Whisper] Fix slow test
* update
* update
* update
* update
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2024-01-10 22:35:36 +01:00
Ella Charlaix
39acfe84ba
Add deepspeed test to amd scheduled CI ( #27633 )
...
* add deepspeed scheduled test for amd
* fix image
* add dockerfile
* add comment
* enable tests
* trigger
* remove trigger for this branch
* trigger
* change runner env to trigger the docker build image test
* use new docker image
* remove test suffix from docker image tag
* replace test docker image with original image
* push new image
* Trigger
* add back amd tests
* fix typo
* add amd tests back
* fix
* comment until docker image build scheduled test fix
* remove deprecated deepspeed build option
* upgrade torch
* update docker & make tests pass
* Update docker/transformers-pytorch-deepspeed-amd-gpu/Dockerfile
* fix
* tmp disable test
* precompile deepspeed to avoid timeout during tests
* fix comment
* trigger deepspeed tests with new image
* comment tests
* trigger
* add sklearn dependency to fix slow tests
* enable back other tests
* final update
---------
Co-authored-by: Felix Marty <felix@hf.co>
Co-authored-by: Félix Marty <9808326+fxmarty@users.noreply.github.com>
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-12-11 16:33:36 +01:00
Yih-Dar
9f1f11a2e7
Show new failing tests in a more clear way in slack report ( #27881 )
...
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-12-07 15:09:30 +01:00
Yih-Dar
64e21ca2a4
Make some jobs run on the GitHub Actions runners ( #27512 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-11-15 10:43:16 +01:00
Yih-Dar
00dc856233
At most 2 GPUs for CI ( #27435 )
...
At most 2 GPUs
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-11-10 16:19:06 +01:00
Yih-Dar
9dc4ce9ea7
Disable CI runner check ( #27170 )
...
Disable runner check
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-10-31 11:59:21 +01:00
Yih-Dar
6ae71ec836
Update runs-on in workflow files ( #26435 )
...
* update
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-09-27 19:25:52 +02:00
Yih-Dar
11cb6e0f7e
Unpin DeepSpeed and require DS >= 0.9.3 ( #24541 )
...
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-06-28 14:01:22 +02:00
Yih-Dar
7631db0fdc
Pin deepspeed to 0.9.2 for now ( #24024 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-06-05 20:00:28 +02:00
Yih-Dar
e69feab8a1
Update workflow files ( #23658 )
...
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-05-22 21:26:51 +02:00
Yih-Dar
aa4316757d
Change schedule CI time ( #22884 )
...
* fix
* Update .github/workflows/self-nightly-past-ci-caller.yml
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
2023-04-20 14:01:08 +02:00
Yih-Dar
648bd5a8aa
Show diff between 2 CI runs on Slack reports ( #22798 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-04-19 19:27:37 +02:00
Yih-Dar
656d41ab4c
Remove DS_BUILD_AIO=1 ( #22741 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-04-13 18:08:22 +02:00
Yih-Dar
0fe6c6bdca
(Re-)Enable Nightly + Past CI ( #22393 )
...
* Enable Nightly + Past CI
* put schedule
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-03-30 21:06:35 +02:00
Yih-Dar
aab895c396
Make Slack CI reporting stronger ( #21823 )
...
* Use token
* Avoid failure
* better error
* Fix
* fix style
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-02-28 17:12:44 +01:00
Yih-Dar
bf9a5882a7
Update some GH action versions ( #20537 )
...
* update actions versions
* update actions versions
* update actions versions
* update actions versions
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-12-06 16:54:40 +01:00
Yih-Dar
67d32f4649
Replace set-output by $GITHUB_OUTPUT ( #20547 )
...
* remove set-output
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-12-05 18:25:13 +01:00
Yih-Dar
e8d448edcf
extract warnings in GH workflows ( #20487 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-11-29 15:58:54 +01:00
Yih-Dar
700e0cd65f
Add missing report button for Example test ( #20293 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-11-17 15:55:00 +01:00
Yih-Dar
c06d555647
Show installed libraries and their versions in GA jobs ( #20069 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-11-04 18:03:18 +01:00
Sylvain Gugger
9ac586b3c8
Rework pipeline tests ( #19366 )
...
* Rework pipeline tests
* Try to fix Flax tests
* Try to put it before
* Use a new decorator instead
* Remove ignore marker since it doesn't work
* Filter pipeline tests
* Woopsie
* Use the fitlered list
* Clean up and fake modif
* Remove init
* Revert fake modif
2022-10-07 18:01:58 -04:00
Yih-Dar
ba7f2173cc
Add runner availability check ( #19054 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-09-19 12:27:06 +02:00
Yih-Dar
7a8118947f
Add checks for more workflow jobs ( #18905 )
...
* add check for scheduled CI
* Add check to other CIs
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-09-07 12:51:37 +02:00
Yih-Dar
7d5fde991d
unpin slack_sdk version ( #18901 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-09-06 18:42:00 +02:00
Yih-Dar
ecdf9b06bc
Remove cached torch_extensions on CI runners ( #18868 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-09-02 18:17:58 +02:00
Yih-Dar
0ab465a5d2
pin Slack SDK to 3.18.1 to avoid failing issue ( #18869 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-09-02 16:49:08 +02:00
Yih-Dar
d2704c4143
Add machine type in the artifact of Examples directory job ( #18459 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-08-04 18:52:01 +02:00
Yih-Dar
ca169dbdf1
Enable PyTorch nightly build CI ( #17335 )
...
* nightly build pytorch CI
* fix working dir
* change time and event name
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-06-17 16:42:27 +02:00
Yih-Dar
da0bed5f4a
Pre-build DeepSpeed ( #17607 )
...
* pre-build deepspeed
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-06-09 23:02:33 +02:00
Yih-Dar
9aa230aa2f
Use latest stable PyTorch/DeepSpeed for Push & Scheduled CI ( #17417 )
...
* update versions
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-06-07 11:53:05 +02:00
Yih-Dar
659b27fd26
Print more library versions in CI ( #17384 )
...
* print more lib. versions and just befor test runs
* update print_env_pt.py
* rename to print_env
* Disable warning + better job name
* print python version
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-06-02 10:24:16 +02:00
Yih-Dar
1ac2b8fa7f
Fix missing job action button in CI report ( #17270 )
...
* use matrix.machine_type
* fix job names used in job_link
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-05-17 08:31:06 +02:00
Yih-Dar
506899d147
Fix push CI channel ( #17242 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-05-13 20:59:56 +02:00
Yih-Dar
993553b2f1
fix --gpus option for docker ( #17235 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-05-13 17:26:26 +02:00
Yih-Dar
38043d8453
Update self-push workflow ( #17177 )
...
* update push ci
* install git-python
* update comment
* update deepspeed jobs
* fix report
* skip 2 more tests that require fairscale
* Fix changes in test_fetcher.py (to deal with `setup.py` is changed)
* set RUN_PT_TF_CROSS_TESTS=1 and final clean-up
* remove SIGOPT_API_TOKEN
* remove echo "$matrix_folders"
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-05-13 16:28:00 +02:00
Yih-Dar
3212afa614
split single_gpu and multi_gpu ( #17083 )
...
* split single_gpu and multi_gpu
* update needs in send_result
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-05-09 07:13:07 -04:00
Yih-Dar
19420fd99e
Move test model folders ( #17034 )
...
* move test model folders (TODO: fix imports and others)
* fix (potentially partially) imports (in model test modules)
* fix (potentially partially) imports (in tokenization test modules)
* fix (potentially partially) imports (in feature extraction test modules)
* fix import utils.test_modeling_tf_core
* fix path ../fixtures/
* fix imports about generation.test_generation_flax_utils
* fix more imports
* fix fixture path
* fix get_test_dir
* update module_to_test_file
* fix get_tests_dir from wrong transformers.utils
* update config.yml (CircleCI)
* fix style
* remove missing imports
* update new model script
* update check_repo
* update SPECIAL_MODULE_TO_TEST_MAP
* fix style
* add __init__
* update self-scheduled
* fix add_new_model scripts
* check one way to get location back
* python setup.py build install
* fix import in test auto
* update self-scheduled.yml
* update slack notification script
* Add comments about artifact names
* fix for yolos
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-05-03 14:42:02 +02:00
Stas Bekman
3a16ab25c8
[self-scheduled ci] explain where dependencies are ( #16757 )
2022-04-13 12:28:02 -04:00
Yih-Dar
09013efdf1
Update step name ( #16189 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-03-16 11:19:38 -04:00
Lysandre Debut
a01fe4cd32
Rebuild deepspeed ( #16081 )
...
* Rebuild deepspeed
* Apply suggestions from code review
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
2022-03-11 14:35:48 -05:00
Lysandre Debut
54f0db4066
Add PT + TF automatic builds ( #15860 )
...
* Add PT + TF automatic builds
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* Wrap up
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2022-03-01 08:55:11 -05:00
Lysandre
c475f3ce2d
Scheduled tests should only run on a daily basis
2022-02-23 17:52:22 -05:00
