riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-31 02:02:21 +06:00
Code Issues Actions 33 Packages Projects Releases Wiki Activity

FIX / OPT: Fix OPT multi-GPU training for OPTForQuestionAnswering (#31092)

Browse Source 
Update modeling_opt.py
This commit is contained in:
Younes Belkada 2024-05-28 17:06:00 +02:00 committed by GitHub
parent 94d416f018
commit ab19f907fd
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
src/transformers/models/opt/modeling_opt.py
View File

@ -1427,8 +1427,8 @@ class OPTForQuestionAnswering(OPTPreTrainedModel):
end_positions = end_positions.squeeze(-1)
# sometimes the start/end positions are outside our model inputs, we ignore these terms
ignored_index = start_logits.size(1)
start_positions = start_positions.clamp(0, ignored_index)
end_positions = end_positions.clamp(0, ignored_index)
start_positions = start_positions.clamp(0, ignored_index).to(logits.device)
end_positions = end_positions.clamp(0, ignored_index).to(logits.device)
loss_fct = CrossEntropyLoss(ignore_index=ignored_index)
start_loss = loss_fct(start_logits, start_positions)