riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-18 20:18:24 +06:00
Code Issues Actions 23 Packages Projects Releases Wiki Activity
4,074 Commits 875 Branches 227 Tags 2 GiB
fb8f4277b2
Commit Graph

8 Commits

Author SHA1 Message Date
Lysandre Debut
6a17688021
per_device instead of per_gpu/error thrown when argument unknown (#4618) 
* per_device instead of per_gpu/error thrown when argument unknown

* [docs] Restore examples.md symlink

* Correct absolute links so that symlink to the doc works correctly

* Update src/transformers/hf_argparser.py

Co-authored-by: Julien Chaumond <chaumond@gmail.com>

* Warning + reorder

* Docs

* Style

* not for squad

Co-authored-by: Julien Chaumond <chaumond@gmail.com>
 2020-05-27 11:36:55 -04:00
Julien Chaumond
5e7fe8b585
Distributed eval: SequentialDistributedSampler + gather all results (#4243) 
* Distributed eval: SequentialDistributedSampler + gather all results

* For consistency only write to disk from world_master

Close https://github.com/huggingface/transformers/issues/4272

* Working distributed eval

* Hook into scripts

* Fix #3721 again

* TPU.mesh_reduce: stay in tensor space

Thanks @jysohn23

* Just a small comment

* whitespace

* torch.hub: pip install packaging

* Add test scenarii
 2020-05-18 22:02:39 -04:00
Lysandre Debut
edf9ac11d4
Should return overflowing information for the log (#4385) 2020-05-15 09:49:11 -04:00
Julien Chaumond
c547f15a17 Use Filelock to ensure distributed barriers 
see context in https://github.com/huggingface/transformers/pull/4223
 2020-05-14 11:58:32 -04:00
Viktor Alm
e4512aab3b
Add MultipleChoice to TFTrainer [WIP] (#4270) 
* catch gpu len 1 set to gpu0

* Add mpc to trainer

* Add MPC for TF

* fix TF automodel for MPC and add Albert

* Apply style

* Fix import

* Note to self: double check

* Make shape None, None for datasetgenerator output shapes

* Add from_pt bool which doesnt seem to work

* Original checkpoint dir

* Fix docstrings for automodel

* Update readme and apply style

* Colab should probably not be from users

* Colabs should probably not be from users

* Add colab

* Update README.md

* Update README.md

* Cleanup __intit__

* Cleanup flake8 trailing comma

* Update src/transformers/training_args_tf.py

* Update src/transformers/modeling_tf_auto.py

Co-authored-by: Viktor Alm <viktoralm@pop-os.localdomain>
Co-authored-by: Julien Chaumond <chaumond@gmail.com>
 2020-05-12 08:48:48 -04:00
Julien Chaumond
7b75aa9fa5
[TPU] Doc, fix xla_spawn.py, only preprocess dataset once (#4223) 
* [TPU] Doc, fix xla_spawn.py, only preprocess dataset once

* Update examples/README.md

* [xla_spawn] Add `_mp_fn` to other Trainer scripts

* [TPU] Fix: eval dataloader was None
 2020-05-08 14:10:05 -04:00
Julien Chaumond
c99fe0386b [doc] Fix broken links + remove crazy big notebook 2020-05-07 18:44:18 -04:00
Julien Chaumond
0ae96ff8a7 BIG Reorganize examples (#4213) 
* Created using Colaboratory

* [examples] reorganize files

* remove run_tpu_glue.py as superseded by TPU support in Trainer

* Bugfix: int, not tuple

* move files around
 2020-05-07 13:48:44 -04:00