riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-31 02:02:21 +06:00
Code Issues Actions 33 Packages Projects Releases Wiki Activity

Fix: Cast prefetch_bucket_size to integer for deepspeed >= 0.15 (#33402)

Browse Source 
Fix: Cast prefetch bucket size to integer in zero_optimization
This commit is contained in:
daejin 2024-09-11 21:25:48 +09:00 committed by GitHub
parent 7a51cbc65f
commit ecf7024bde
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
src/transformers/integrations/deepspeed.py
View File

@ -241,7 +241,7 @@ class HfTrainerDeepSpeedConfig(HfDeepSpeedConfig):
# automatically assign the optimal config values based on model config
self.fill_only(
"zero_optimization.stage3_prefetch_bucket_size",
0.9 * hidden_size * hidden_size,
int(0.9 * hidden_size * hidden_size),
)
self.fill_only(
"zero_optimization.stage3_param_persistence_threshold",