Joao Gante
0863eef248
[tests] remove pt_tf equivalence tests ( #36253 )
2025-02-19 11:55:11 +00:00
Guillaume LEGENDRE
4df3ccddb7
Migrate the CI runners to the new clusters ( #33849 )
...
* try fixing push-ci
* move to new runners
* move benchmark.yml to new runners
* move doctest_job.yml to new runners
* move doctests.yml to new runners
* move push-important-models.yml to new runners
* move self-pr-slow-ci.yml to new runners
* fix typo
Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>
* fix working directory
Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>
* fix working directory
Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>
* improve code
Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>
---------
Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>
2024-10-03 14:39:49 +02:00
Yih-Dar
3d79dcbda0
update push CI workflow files for security ( #33142 )
...
* update for security 1
* update for security 2
* update for security 3
* update for security 4
* update for security 5
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2024-08-28 18:15:58 +02:00
Sai-Suraj-27
af638c4afe
fix: Added missing huggingface_hub installation to workflows ( #32891 )
...
Added missing huggingface_hub installation to workflows.
2024-08-22 12:51:12 +01:00
Yih-Dar
fbb41cd420
consistent job / pytest report / artifact name correspondence ( #30392 )
...
* better names
* run better names
* update
* update
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2024-04-24 22:32:42 +02:00
Yih-Dar
440bd3c3c0
update github actions packages' version to suppress warnings ( #30249 )
...
update
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2024-04-15 15:08:09 +02:00
Yih-Dar
95346e9dcd
Add artifact name in job step to maintain job / artifact correspondence ( #28682 )
...
* avoid using job name
* apply to other files
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2024-01-31 15:58:17 +01:00
Ella Charlaix
39acfe84ba
Add deepspeed test to amd scheduled CI ( #27633 )
...
* add deepspeed scheduled test for amd
* fix image
* add dockerfile
* add comment
* enable tests
* trigger
* remove trigger for this branch
* trigger
* change runner env to trigger the docker build image test
* use new docker image
* remove test suffix from docker image tag
* replace test docker image with original image
* push new image
* Trigger
* add back amd tests
* fix typo
* add amd tests back
* fix
* comment until docker image build scheduled test fix
* remove deprecated deepspeed build option
* upgrade torch
* update docker & make tests pass
* Update docker/transformers-pytorch-deepspeed-amd-gpu/Dockerfile
* fix
* tmp disable test
* precompile deepspeed to avoid timeout during tests
* fix comment
* trigger deepspeed tests with new image
* comment tests
* trigger
* add sklearn dependency to fix slow tests
* enable back other tests
* final update
---------
Co-authored-by: Felix Marty <felix@hf.co>
Co-authored-by: Félix Marty <9808326+fxmarty@users.noreply.github.com>
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-12-11 16:33:36 +01:00
Yih-Dar
64e21ca2a4
Make some jobs run on the GitHub Actions runners ( #27512 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-11-15 10:43:16 +01:00
Yih-Dar
00dc856233
At most 2 GPUs for CI ( #27435 )
...
At most 2 GPUs
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-11-10 16:19:06 +01:00
Yih-Dar
9dc4ce9ea7
Disable CI runner check ( #27170 )
...
Disable runner check
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-10-31 11:59:21 +01:00
Yih-Dar
6ae71ec836
Update runs-on in workflow files ( #26435 )
...
* update
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-09-27 19:25:52 +02:00
Yih-Dar
e69feab8a1
Update workflow files ( #23658 )
...
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-05-22 21:26:51 +02:00
Yih-Dar
656d41ab4c
Remove DS_BUILD_AIO=1 ( #22741 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-04-13 18:08:22 +02:00
Yih-Dar
aab895c396
Make Slack CI reporting stronger ( #21823 )
...
* Use token
* Avoid failure
* better error
* Fix
* fix style
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-02-28 17:12:44 +01:00
Yih-Dar
bf9a5882a7
Update some GH action versions ( #20537 )
...
* update actions versions
* update actions versions
* update actions versions
* update actions versions
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-12-06 16:54:40 +01:00
Yih-Dar
67d32f4649
Replace set-output by $GITHUB_OUTPUT ( #20547 )
...
* remove set-output
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-12-05 18:25:13 +01:00
Yih-Dar
c06d555647
Show installed libraries and their versions in GA jobs ( #20069 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-11-04 18:03:18 +01:00
Yih-Dar
e81cb010f8
Avoid Push CI failing to report due to many commits being merged ( #19496 )
...
* Change the depth to 20
* Add comment
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-10-12 09:25:05 +02:00
Yih-Dar
8d68878cc0
python3 instead of python in push CI setup job (#19492 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-10-11 19:18:39 +02:00
Yih-Dar
99c32493e0
Fix confusing working directory in Push CI ( #19234 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-09-29 08:36:46 +02:00
Yih-Dar
71fc331746
Separate Push CI images from Scheduled CI ( #19170 )
...
* separate images
* Fix condition
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-09-26 10:55:42 +02:00
Yih-Dar
22264f933d
fix working dir ( #19101 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-09-19 07:09:24 -04:00
Yih-Dar
ba7f2173cc
Add runner availability check ( #19054 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-09-19 12:27:06 +02:00
Yih-Dar
7d5fde991d
unpin slack_sdk version ( #18901 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-09-06 18:42:00 +02:00
Yih-Dar
ecdf9b06bc
Remove cached torch_extensions on CI runners ( #18868 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-09-02 18:17:58 +02:00
Yih-Dar
0ab465a5d2
pin Slack SDK to 3.18.1 to avoid failing issue ( #18869 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-09-02 16:49:08 +02:00
Yih-Dar
81ab11124f
Add checks for some workflow jobs ( #18583 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-08-16 13:53:47 +02:00
Yih-Dar
8d634b70e0
Fix push CI artifact path ( #17788 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-06-23 12:31:22 +02:00
Yih-Dar
6589e510fa
Attempt to change Push CI to workflow_run ( #17753 )
...
* Use workflow_run event for push CI
* change to workflow_run
* Add comments
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-06-18 08:35:03 +02:00
Yih-Dar
ca169dbdf1
Enable PyTorch nightly build CI ( #17335 )
...
* nightly build pytorch CI
* fix working dir
* change time and event name
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-06-17 16:42:27 +02:00
Yih-Dar
50415b84d6
Revert "Change push CI to run on workflow_run event ( #17692 )" ( #17717 )
...
This reverts commit b76290f44c
2022-06-15 18:42:43 +02:00
Yih-Dar
b76290f44c
Change push CI to run on workflow_run event ( #17692 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-06-15 17:43:31 +02:00
Yih-Dar
da0bed5f4a
Pre-build DeepSpeed ( #17607 )
...
* pre-build deepspeed
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-06-09 23:02:33 +02:00
Yih-Dar
9aa230aa2f
Use latest stable PyTorch/DeepSpeed for Push & Scheduled CI ( #17417 )
...
* update versions
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-06-07 11:53:05 +02:00
Yih-Dar
659b27fd26
Print more library versions in CI ( #17384 )
...
* print more lib. versions and just befor test runs
* update print_env_pt.py
* rename to print_env
* Disable warning + better job name
* print python version
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-06-02 10:24:16 +02:00
Yih-Dar
0932adb3e8
Split push CI into 2 workflows ( #17369 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-06-02 10:19:26 +02:00
Yih-Dar
1ac2b8fa7f
Fix missing job action button in CI report ( #17270 )
...
* use matrix.machine_type
* fix job names used in job_link
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-05-17 08:31:06 +02:00
Yih-Dar
50d1867cf8
Add PR title to push CI report ( #17246 )
...
* add PR title to push CI report
* add link
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-05-13 21:50:40 +02:00
Yih-Dar
506899d147
Fix push CI channel ( #17242 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-05-13 20:59:56 +02:00
Yih-Dar
38043d8453
Update self-push workflow ( #17177 )
...
* update push ci
* install git-python
* update comment
* update deepspeed jobs
* fix report
* skip 2 more tests that require fairscale
* Fix changes in test_fetcher.py (to deal with `setup.py` is changed)
* set RUN_PT_TF_CROSS_TESTS=1 and final clean-up
* remove SIGOPT_API_TOKEN
* remove echo "$matrix_folders"
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-05-13 16:28:00 +02:00
Yih-Dar
351cdbdfdc
Fix self-push CI report path in cat ( #17111 )
...
* fix report cat path
* fix report cat path
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-05-06 07:45:17 -07:00
Sylvain Gugger
4be8b95a9f
Disable Flax GPU tests on push ( #17042 )
2022-05-02 10:25:53 -04:00
Stas Bekman
ce2fef2ad2
[trainer / deepspeed] fix hyperparameter_search ( #16740 )
...
* [trainer / deepspeed] fix hyperparameter_search
* require optuna
* style
* oops
* add dep in the right place
* create deepspeed-testing dep group
* Trigger CI
2022-04-14 17:24:38 -07:00
Lysandre Debut
eca77f4719
Updates the default branch from master to main ( #16326 )
...
* Updates the default branch from master to main
* Links from `master` to `main`
* Typo
* Update examples/flax/README.md
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2022-03-23 03:46:59 -04:00
Suraj Patil
afca0d5192
use python 3.7 for flax self-push tests ( #15865 )
...
* set python 3.7 for flax tests
* setup-python@v2
* python-dev
* install -y
* python3-dev
* install kenlm from source
* install cython
* cd to kenlm
* kenlm install
* don't install kenlm
* change flax pretrained to run flax tests
* cleanup
* remove python-dev
2022-03-01 18:26:30 +01:00
Lysandre Debut
b7e292aebd
Fix the push run ( #15807 )
2022-02-24 19:30:17 +01:00
Patrick von Platen
c4a96cecbc
Wav2Vec2 meets phonemes ( #14353 )
...
* up
* add tokenizer
* improve more
* finish tokenizer
* finish
* adapt speech recognition script
* adapt convert
* more fixes
* more fixes
* update phonemizer wav2vec2
* better naming
* fix more tests
* more fixes swedish
* correct tests
* finish
* improve script
* remove file
* up
* lets get those 100 model architectures until the end of the month
* make fix-copies
* correct more
* correct script
* more fixes
* more fixes
* add to docs
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* replace assert
* fix copies
* fix docs
* new try docs
* boom boom
* update
* add phonemizer to audio tests
* make fix-copies
* up
* upload models
* some changes
* Update tests/test_tokenization_wav2vec2_phoneme.py
Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
* more fixes
* remove @
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
2021-12-17 19:56:44 +01:00
Patrick von Platen
961732c276
[Wav2Vec2] PyCTCDecode Integration to support language model boosted decoding ( #14339 )
...
* up
* up
* up
* make it cleaner
* correct
* make styhahalal
* add more tests
* finish
* small fix
* make style
* up
* tryout to solve cicrle ci
* up
* fix more tests
* fix more tests
* apply sylvains suggestions
* fix import
* correct docs
* add pyctcdecode only to speech tests
* fix more tests
* add tf, flax and pt tests
* add pt
* fix last tests
* fix more tests
* Apply suggestions from code review
* change lines
* Apply suggestions from code review
Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
* correct tests
* correct tests
* add doc string
Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
2021-12-08 12:07:54 +01:00
Stas Bekman
e62091d5a7
[CI] move env print to util, add pt, nccl versions ( #14607 )
...
* move env print to util, add pt, nccl versions
* style
* version
* align
2021-12-03 08:18:36 -05:00
