riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-31 02:02:21 +06:00
Code Issues Actions 33 Packages Projects Releases Wiki Activity

fsdp bf16 enable autocast (#21847)

Browse Source
This commit is contained in:
Sourab Mangrulkar 2023-03-02 20:18:07 +05:30 committed by GitHub
parent fb76994c41
commit b6f47b5393
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 0 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
src/transformers/trainer.py
View File

@ -618,9 +618,6 @@ class Trainer:
self.scaler = GradScaler()
else:
self.scaler = torch.cuda.amp.GradScaler()
elif self.fsdp is not None:
self.use_cuda_amp = False
self.amp_dtype = None
elif args.half_precision_backend == "cpu_amp":
self.use_cpu_amp = True
self.amp_dtype = torch.bfloat16