mirror of
https://github.com/huggingface/transformers.git
synced 2025-07-31 02:02:21 +06:00
4868a830db
* add simple multi gpu complet * add human_eval_multi_gpu * use copy strategy to distribute across gpu, to avoid padding * add doc string * update code style * use task id to arrange output * truncate input to avoid zero pad * Stop the copy mechanism * update style * restore copies to scale better in distributed mode * update style * replace human eval * Apply suggestions from code review 1. Tokenize all input at the same time 2. use attention_mask to get the input length 3. other small fixes Co-authored-by: Leandro von Werra <lvwerra@users.noreply.github.com> * correct typo and update docstring * update code style * remove num sample division constraint * remove max len calculation * use accelerator.gather once to speed up * use accelerate set_seed; update accelerate version * correct gather bug Co-authored-by: Leandro von Werra <lvwerra@users.noreply.github.com>
7 lines
124 B
Plaintext
7 lines
124 B
Plaintext
transformers==4.15.0
|
|
datasets==1.16.0
|
|
accelerate==0.6.2
|
|
wandb==0.12.0
|
|
tensorboard==2.6.0
|
|
torch==1.9.0
|
|
huggingface-hub==0.1.0 |