riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-04 05:10:06 +06:00
Code Issues Actions 20 Packages Projects Releases Wiki Activity

Add adam_kwargs for Apollo Optimizer (#38168)

Browse Source 
Add adam_kwargs for Apollo
This commit is contained in:
Daize Dong 2025-05-19 16:59:49 +08:00 committed by GitHub
parent 6bb6821d93
commit 656e2eab3f
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
src/transformers/trainer.py
View File

@ -1615,6 +1615,7 @@ class Trainer:
"scale": float(optim_args.pop("scale", 1.0)),
"proj_type": optim_args.pop("proj_type", "std"),
}
apollo_optim_kwargs.update(adam_kwargs)
optimizer_cls, optimizer_kwargs = setup_low_rank_optimizer(
args.optim, optimizer_mapping, apollo_optim_kwargs