riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-31 02:02:21 +06:00
Code Issues Actions 33 Packages Projects Releases Wiki Activity

Drop inplace operation for loss computation with gradient accumulation (#35416)

Browse Source 
Fix inplace loss computation
This commit is contained in:
Quentin Gallouédec 2024-12-26 14:58:53 +01:00 committed by GitHub
parent 24c91f095f
commit 4eb17b26e7
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
src/transformers/trainer.py
View File

@ -3700,7 +3700,7 @@ class Trainer:
else:
# Finally we need to normalize the loss for reporting
if num_items_in_batch is None:
loss /= self.args.gradient_accumulation_steps
loss = loss / self.args.gradient_accumulation_steps
self.accelerator.backward(loss, **kwargs)