Yih-Dar
10627c1a0f
Use torch 2.7.1 on daily CI ( #38620 )
...
Self-hosted runner (benchmark) / Benchmark (aws-g5-4xlarge-cache) (push) Waiting to run
Build documentation / build (push) Waiting to run
Slow tests on important models (on Push - A10) / Get all modified files (push) Waiting to run
Slow tests on important models (on Push - A10) / Slow & FA2 tests (push) Blocked by required conditions
Secret Leaks / trufflehog (push) Waiting to run
Update Transformers metadata / build_and_package (push) Waiting to run
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2025-06-08 14:37:45 +02:00
Yih-Dar
4143f94d51
uninstall kernels from docker images ( #38083 )
...
uninstall kernels
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2025-05-12 18:03:47 +02:00
Marc Sun
7ae0be722e
update deepspeed docker ( #37371 )
...
* update
* create docker image
* 03
* uninstall pytest as it conflits with transformers
* wrong one
* better
* see which package depends on pytest
* up
* resintall
* fix
* deepspeedddddddd
* deepspeedddddddd
* deepspeedddddddd
* deepspeedddddddd
* deepspeedddddddd
* deepspeedddddddd
* deepspeedddddddd
* deepspeedddddddd
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2025-04-09 14:54:06 +02:00
Yih-Dar
ec7790f0d3
update docker file transformers-pytorch-deepspeed-latest-gpu ( #35940 )
...
update docker file for deepspeed
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2025-01-29 16:01:27 +01:00
Sai-Suraj-27
3562772969
fix: Fixed pydantic required version in dockerfiles to make it compatible with DeepSpeed ( #33105 )
...
Fixed pydantic required version in dockerfiles.
2024-08-26 17:10:36 +02:00
Zach Mueller
0cea2081a3
Unpin deepspeed in Docker image/tests ( #32572 )
...
Unpin deepspeed
2024-08-14 18:30:25 +01:00
Zach Mueller
5b7a225f25
Pin deepspeed ( #30701 )
...
pin ds
2024-05-07 13:45:24 -04:00
Yih-Dar
5c341d4555
Use torch 2.2 for deepspeed CI ( #29246 )
...
update
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2024-02-27 17:51:37 +08:00
Ella Charlaix
39acfe84ba
Add deepspeed test to amd scheduled CI ( #27633 )
...
* add deepspeed scheduled test for amd
* fix image
* add dockerfile
* add comment
* enable tests
* trigger
* remove trigger for this branch
* trigger
* change runner env to trigger the docker build image test
* use new docker image
* remove test suffix from docker image tag
* replace test docker image with original image
* push new image
* Trigger
* add back amd tests
* fix typo
* add amd tests back
* fix
* comment until docker image build scheduled test fix
* remove deprecated deepspeed build option
* upgrade torch
* update docker & make tests pass
* Update docker/transformers-pytorch-deepspeed-amd-gpu/Dockerfile
* fix
* tmp disable test
* precompile deepspeed to avoid timeout during tests
* fix comment
* trigger deepspeed tests with new image
* comment tests
* trigger
* add sklearn dependency to fix slow tests
* enable back other tests
* final update
---------
Co-authored-by: Felix Marty <felix@hf.co>
Co-authored-by: Félix Marty <9808326+fxmarty@users.noreply.github.com>
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-12-11 16:33:36 +01:00
Zach Mueller
acd653164b
Update CUDA versions for DeepSpeed ( #27853 )
...
* Update CUDA versions
* For testing
* Allow for workflow dispatch
* Use newer image
* Revert workflow
* Revert workflow
* Push
* Other docker image
2023-12-05 16:15:21 -05:00
Yih-Dar
b219ae6bd4
Update docker files to use torch==2.1.0 ( #26735 )
...
Update docker files to use torch 2.1
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-10-11 16:23:36 +02:00
ydshieh
66ded238cd
fix pydantic install command
2023-07-01 09:29:21 +02:00
Serge Matveenko
d51aa48a76
Limit Pydantic to V1 in dependencies ( #24596 )
...
* Limit Pydantic to V1 in dependencies
Pydantic is about to release V2 release which will break a lot of things. This change prevents `transformers` to be used with Pydantic V2 to avoid breaking things.
* more
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-07-01 00:04:03 +02:00
Yih-Dar
17e3e7d686
pin apex to a speicifc commit (for DeepSpeed CI docker image) ( #24351 )
...
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-06-19 12:48:53 +02:00
Yih-Dar
db4d765249
Fix transformers' DeepSpeed CI job ( #23463 )
...
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-05-19 17:50:06 +02:00
Yih-Dar
22a0769933
Update 3 docker files to use cu118 ( #23406 )
...
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-05-17 14:26:50 +02:00
Yih-Dar
ba71d9e94c
unpin tf prob ( #23293 )
...
* unpin tf prob
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-05-11 21:28:08 +02:00
Yih-Dar
4603fe9b1f
use accelerate@main in CI ( #22859 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-04-19 14:58:53 +02:00
Yih-Dar
656d41ab4c
Remove DS_BUILD_AIO=1 ( #22741 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-04-13 18:08:22 +02:00
Yih-Dar
01203475c9
Update docker files to use official torch 2.0.0 ( #22357 )
...
* update docker files to use official torch 2.0.0
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-03-24 14:29:05 +01:00
Yih-Dar
bec075612a
Revert "Use dash==2.8.1 for now for daily CI" ( #22233 )
...
Revert "Use `dash==2.8.1` for now for daily CI (#22227 )"
This reverts commit 53218671d9
2023-03-17 16:54:27 +01:00
Yih-Dar
53218671d9
Use dash==2.8.1 for now for daily CI ( #22227 )
...
Use dash 2.8.1 for now
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-03-17 13:27:14 +01:00
Yih-Dar
1c4a9acc73
Fix DeepSpeed CI ( #22194 )
...
* Deal with torch-tensorrt
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-03-16 05:52:40 +01:00
Yih-Dar
ba9e0191de
Prepare daily CI for torch 2.0.0 ( #22135 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-03-13 22:21:15 +01:00
Yih-Dar
db572b3854
Use torch 1.13.1 in push/schedule CI ( #21421 )
...
Use torch 1.13.1 in push/scheduled CI
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-02-02 14:58:52 +01:00
Yih-Dar
94db82573e
Fix (DeepSpeed) docker image build issue ( #21002 )
...
* Fix docker image build issue
* remove comment
* Add comment
* Update docker/transformers-pytorch-deepspeed-latest-gpu/Dockerfile
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
2023-01-04 21:28:33 +01:00
Yih-Dar
1543cee7c8
Recompile apex in DeepSpeed CI image ( #20788 )
...
Recompile apex in DeepSpeed CI image
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-12-15 21:35:27 +01:00
Yih-Dar
94f8e21c70
Install torch-tensorrt 1.3.0 for DeepSpeed CI ( #20764 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-12-14 17:30:36 +01:00
Yih-Dar
d994473b05
Uninstall torch_tensorrt in DeepSpeed CI image for now ( #20758 )
...
Uninstall torch_tensorrt for now
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-12-13 22:25:47 +01:00
Yih-Dar
d4bf9ee1ff
Update CI to torch 1.13.0 ( #20687 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-12-12 20:04:56 +01:00
Yih-Dar
ecdf9b06bc
Remove cached torch_extensions on CI runners ( #18868 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-09-02 18:17:58 +02:00
Yih-Dar
510c2a0b32
Change scheduled CIs to use torch 1.12.1 ( #18644 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-08-16 13:41:37 +02:00
Yih-Dar
b089cca347
PyTorch 1.12.0 for scheduled CI ( #17949 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-06-29 19:32:19 +02:00
Yih-Dar
9fe2403bc5
Use explicit torch version in deepspeed CI ( #17942 )
...
* use explicit torch version
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-06-29 18:20:34 +02:00
Yih-Dar
ca169dbdf1
Enable PyTorch nightly build CI ( #17335 )
...
* nightly build pytorch CI
* fix working dir
* change time and event name
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-06-17 16:42:27 +02:00
Yih-Dar
da0bed5f4a
Pre-build DeepSpeed ( #17607 )
...
* pre-build deepspeed
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-06-09 23:02:33 +02:00
Yih-Dar
9aa230aa2f
Use latest stable PyTorch/DeepSpeed for Push & Scheduled CI ( #17417 )
...
* update versions
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-06-07 11:53:05 +02:00
Stas Bekman
ce2fef2ad2
[trainer / deepspeed] fix hyperparameter_search ( #16740 )
...
* [trainer / deepspeed] fix hyperparameter_search
* require optuna
* style
* oops
* add dep in the right place
* create deepspeed-testing dep group
* Trigger CI
2022-04-14 17:24:38 -07:00
Sylvain Gugger
867f3950fa
Rename master to main for notebooks links and leftovers ( #16397 )
2022-03-25 09:12:23 -04:00
Lysandre Debut
a0e3480699
[Test refactor 5/5] Build docker images ( #15729 )
2022-02-23 15:48:19 -05:00
