Jeff Rasley
6ebeeeef81
Update requirements.txt ( #17719 )
2022-06-15 13:51:41 -04:00
Sylvain Gugger
3cab90279f
Add examples telemetry ( #17552 )
...
* Add examples telemetry
* Alternative approach
* Add to all other examples
* Add to templates as well
* Put framework separately
* Same for TensorFlow
2022-06-07 11:57:52 -04:00
bhuang
254d9c068e
Update run_glue_no_trainer.py ( #17546 )
2022-06-03 12:29:37 -04:00
Zachary Mueller
3766df4fe1
Fix flakey no-trainer test ( #17515 )
2022-06-01 13:40:49 -04:00
fireindark707
028d4b7c8b
Deal with the error when task is regression ( #16330 )
2022-06-01 11:15:53 -04:00
Sourab Mangrulkar
d156898f3b
Improve notrainer examples ( #17449 )
...
* improve no-trainer examples
* Trigger CI
* adding comment to clarify tracker init on main process
* Trigger CI
* Trigger CI
* Trigger CI
2022-05-28 00:06:31 +05:30
Patrick von Platen
a9eca74372
Wav2vec2 finetuning shared file system ( #17423 )
...
* fix_torch_device_generate_test
* remove @
* [Fix shared file system]
Co-authored-by: Patrick von Platen <patrick@huggingface.co>
2022-05-25 22:04:43 +02:00
Zachary Mueller
1762ded30a
Fix metric calculation in examples and setup tests to run on multi-gpu for no_trainer scripts ( #17331 )
...
* Fix length in no_trainer examples
* Add setup and teardown
* Use new accelerator config generator to automatically make tests able to run based on environment
2022-05-18 14:17:40 -04:00
regisss
28a0811652
Improve mismatched sizes management when loading a pretrained model ( #17257 )
...
- Add --ignore_mismatched_sizes argument to classification examples
- Expand the error message when loading a model whose head dimensions are different from expected dimensions
2022-05-17 17:58:14 +02:00
Sylvain Gugger
afe5d42d8d
Black preview ( #17217 )
...
* Black preview
* Fixup too!
* Fix check copies
* Use the same version as the CI
* Bump black
2022-05-12 16:25:55 -04:00
Lysandre Debut
5294fa12ee
Dev version
2022-05-12 11:04:23 -04:00
Zachary Mueller
d719bcd46a
Fix all docs for accelerate install directions ( #17145 )
2022-05-09 15:45:18 -04:00
Zachary Mueller
ef20390291
Update to build via git for accelerate ( #17084 )
2022-05-04 09:42:36 -04:00
Zachary Mueller
f275e593bf
Fix no_trainer examples to properly calculate the number of samples ( #17046 )
...
* Update all examples to properly calculate progress bar
2022-05-02 11:56:25 -04:00
Zachary Mueller
35d48db881
Update no_trainer examples to use new logger ( #17044 )
...
* Propagate and fix imports
2022-05-02 11:56:15 -04:00
yujun
bdd690a74d
add torch.no_grad when in eval mode ( #17020 )
...
* add torch.no_grad when in eval mode
* make style quality
2022-05-02 07:49:19 -04:00
Zachary Mueller
3486a92a57
Fix savedir for by epoch ( #16996 )
2022-04-28 13:49:45 -04:00
Zachary Mueller
60e1d883f1
Fixup no_trainer save logic ( #16968 )
...
* Fixup all examples
2022-04-27 14:46:49 -04:00
Sylvain Gugger
c79bbc3ba5
Fix multiple deletions of the same files in save_pretrained ( #16947 )
...
* Fix multiple deletions of the same files in save_pretrained
* Add is_main_process argument
2022-04-27 12:28:42 -04:00
Leonid Boytsov
c82e017aa9
Misc. fixes for Pytorch QA examples: ( #16958 )
...
1. Fixes evaluation errors popping up when you train/eval on squad v2 (one was newly encountered and one that was previously reported Running SQuAD 1.0 sample command raises IndexError #15401 but not completely fixed).
2. Removes boolean arguments that don't use store_true. Please, don't use these: *ANY non-empty string is being converted to True in this case and this clearly is not the desired behavior (and it creates a LOT of confusion).
3. All no-trainer test scripts are now saving metric values in the same way (with the right prefix eval_), which is consistent with the trainer-based versions.
4. Adds forgotten model.eval() in the no-trainer versions. This improved some results, but not everything (see the discussion in the end). Please, see the F1 scores and the discussion below.
2022-04-27 08:51:39 -04:00
NielsRogge
479fdc4925
Add semantic script, trainer ( #16834 )
...
* Add first draft
* Improve script and README
* Improve README
* Apply suggestions from code review
* Improve script, add link to resulting model
* Add corresponding test
* Adjust learning rate
2022-04-27 10:12:18 +02:00
Zachary Mueller
705d65368f
Fix multiproc metrics in no_trainer examples ( #16865 )
2022-04-20 17:26:27 -04:00
NielsRogge
b96e82c80a
Add image classification script, no trainer ( #16727 )
...
* Add first draft
* Improve README and run fixup
* Make script aligned with other scripts, improve README
* Improve script and add test
* Remove print statement
* Apply suggestions from code review
* Add num_labels to make test pass
* Improve README
2022-04-19 16:32:08 +02:00
NielsRogge
7db7aab439
Add semantic script no trainer, v2 ( #16788 )
...
* Add first draft from previous PR
* First draft
* Improve README and remove num_labels
* Make script more aligned with other scripts
* Improve README and apply suggestion from code review
2022-04-19 09:07:29 +02:00
NielsRogge
78f346c2b5
Update README.md ( #16797 )
2022-04-15 14:10:16 +02:00
NielsRogge
048443db86
Improve image classification example ( #16585 )
...
* Improve README
* Make dataset_name argument optional
* Improve local data
* Fix bug
* Improve README some more
* Apply suggestions from code review
* Improve README
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
2022-04-14 18:10:52 +02:00
Zachary Mueller
be752d12f8
Fixup no_trainer examples scripts and add more tests ( #16765 )
...
* Change tracking to store_true
* Remove step param and use it in the log dictionary directly
* use vars(args) when passing args to init_trackers
* Include tracking tests since tensorboard is already a dep
2022-04-13 14:40:48 -04:00
Heerak Son
db3edd050b
Update run_translation_no_trainer.py ( #16652 )
...
args.model_name_or_path -> args.config_name
fix it
2022-04-12 08:55:12 -04:00
Zachary Mueller
69233cf03b
Fix example logs repeating themselves ( #16669 )
...
Move declaration of log streams to before tests, so that results won't get compounded on top of each other
2022-04-11 16:25:16 -04:00
Zachary Mueller
d4b3e359aa
Don't push checkpoints to hub in no_trainer scripts ( #16703 )
...
Adds checkpoint prefixes to the gitignore if `push_to_hub` is used along with `checkpointint_steps`
2022-04-11 12:42:45 -04:00
Zachary Mueller
d57da99237
Add tests for no_trainer and fix existing examples ( #16656 )
...
* Fixed some bugs involving saving during epochs
* Added tests mimicking the existing examples tests
* Added in json exporting to all `no_trainer` examples for consistency
2022-04-08 10:03:56 -04:00
Zachary Mueller
febe42b5da
Update no_trainer scripts with new Accelerate functionalities ( #16617 )
...
Adds logging and save/loading to the Accelerate scripts
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2022-04-06 15:29:32 -04:00
Lysandre Debut
a180efe7fd
Dev version
2022-04-06 11:08:12 -04:00
Karim Foda
24a85cca61
Add use_auth to load_datasets for private datasets to PT and TF examples ( #16521 )
...
* fix formatting and remove use_auth
* Add use_auth_token to Flax examples
2022-04-04 10:27:45 -04:00
Bhadresh Savani
05b4c32908
fixed a typo ( #16508 )
2022-03-31 07:49:02 -04:00
Stas Bekman
a73281e3e4
[examples] max samples can't be bigger than the len of dataset ( #16501 )
...
* [examples] max samples can't be bigger than then len of dataset
* do tf and flax
2022-03-30 12:33:16 -07:00
Sylvain Gugger
b62ac4d240
Fix example test and test_fetcher for examples ( #16478 )
2022-03-29 12:21:19 -04:00
Eldar Kurtic
5216607f8a
[MNLI example] Prevent overwriting matched with mismatched metrics ( #16475 )
...
* Prevent overwriting matched with mismatched metrics
* Fix style
2022-03-29 10:38:14 -04:00
Sylvain Gugger
867f3950fa
Rename master to main for notebooks links and leftovers ( #16397 )
2022-03-25 09:12:23 -04:00
Sylvain Gugger
088c1880b7
Big file_utils cleanup ( #16396 )
...
* Big file_utils cleanup
* This one still needs to be treated separately
2022-03-25 07:25:20 -04:00
Sylvain Gugger
4975002df5
Reorganize file utils ( #16264 )
...
* Split file_utils in several submodules
* Fixes
* Add back more objects
* More fixes
* Who exactly decided to import that from there?
* Second suggestion to code with code review
* Revert wront move
* Fix imports
* Adapt all imports
* Adapt all imports everywhere
* Revert this import, will fix in a separate commit
2022-03-23 10:26:33 -04:00
Lysandre Debut
eca77f4719
Updates the default branch from master to main ( #16326 )
...
* Updates the default branch from master to main
* Links from `master` to `main`
* Typo
* Update examples/flax/README.md
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2022-03-23 03:46:59 -04:00
Sylvain Gugger
19597998f6
Don't compute metrics in LM examples on TPU ( #16029 )
2022-03-10 07:44:51 -05:00
Shotaro Ishihara
8feede229c
Fix broken code blocks in README.md ( #15967 )
...
at transformers/examples/pytorch/contrastive-image-text
2022-03-09 17:07:52 +01:00
Joao Gante
e7f34ccd4f
Swag example: Update doc format ( #16014 )
2022-03-09 13:25:34 +00:00
davidleonfdez
c0281feb50
Fix #15898 ( #15928 )
2022-03-03 14:41:03 -05:00
Sylvain Gugger
79d28e80b6
v4.18.0.dev.0
2022-03-03 10:19:58 -05:00
Suraj Patil
bf1fe32824
[examples/summarization and translation] fix readme ( #15833 )
2022-02-25 17:28:16 +01:00
Lysandre Debut
29c10a41d0
[Test refactor 1/5] Per-folder tests reorganization ( #15725 )
...
* Per-folder tests reorganization
Co-authored-by: sgugger <sylvain.gugger@gmail.com>
Co-authored-by: Stas Bekman <stas@stason.org>
2022-02-23 15:46:28 -05:00
Yongrae Jo
3db2e8f92b
Fix typo on examples/pytorch/question-answering ( #15644 )
...
cna -> can
2022-02-22 13:51:07 -05:00
Joao Gante
3956b133b6
TF text classification examples ( #15704 )
...
* Working example with to_tf_dataset
* updated text_classification
* more comments
2022-02-21 17:17:59 +00:00
Suraj Patil
86119c1154
add VisionTextDualEncoder and CLIP fine-tuning script ( #15701 )
...
* begin script
* update script
* fix features and data args
* main
* add requirements
* add column name args
* fix captions
* don't jit transforms
* fix caption
* fix labels, handle attention mask
* convert pixel values to numpy
* labels => input_ids
* transform images on the fly
* use AutoModel class, create the hybird model outside of the script
* fix version message
* add readme
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* adderss review comments
* add more comments
* allow freezing vision and text models
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2022-02-21 16:10:59 +01:00
Simon Sardorf
a63bd3675f
Remove input and target reset after preprocessing ( #15741 )
...
Remove input and target reset after preprocessing
2022-02-21 11:10:15 +01:00
NielsRogge
57882177be
Add SimMIM ( #15586 )
...
* Add first draft
* Make model importable
* Make SwinForMaskedImageModeling importable
* Fix imports
* Add missing inits
* Add support for Swin
* Fix bug
* Fix bug
* Fix another bug
* Fix Swin MIM implementation
* Fix default encoder stride
* Fix Swin
* Add print statements for debugging
* Add image_size data argument
* Fix Swin
* Fix image_size
* Add print statements for debugging
* Fix print statement
* Remove print statements
* Improve reshaping of bool_masked_pos
* Add support for DeiT, fix tests
* Improve docstrings
* Apply new black version
* Improve script
* Fix bug
* Improve README
* Apply suggestions from code review
* Remove DS_Store and add to gitignore
* Apply suggestions from code review + fix BEiT Flax
* Revert BEiT changes
* Improve README
* Fix code quality
* Improve README
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MBP.localdomain>
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
2022-02-17 19:44:55 +01:00
NielsRogge
0e91f885c3
Add image classification notebook ( #15667 )
...
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
2022-02-17 13:14:01 +01:00
Patrick von Platen
3d5dea9bf0
Add example batch size to all commands ( #15596 )
2022-02-10 08:52:07 -05:00
Lysandre Debut
7732d0fe7a
Upgrade black to version ~=22.0 ( #15565 )
...
* Upgrade black to version ~=22.0
* Check copies
* Fix code
2022-02-09 09:28:57 -05:00
Anton Lozhkov
a459f7f97d
Add ASR CTC streaming example ( #15309 )
...
* Single-epoch run
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* Infinite dataset
* Trainer fix + distributed benchmark
* Benchmark fix
* unused import
* interleaved splits
* interleaved splits
* has_length util
* Move to research projects
* Leftover Sized checks
* Bump min version
* Unused import
* Revert trainer changes
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2022-02-07 18:35:37 +03:00
davidleonfdez
f1a4c4ead5
[WIP] Add preprocess_logits_for_metrics Trainer param ( #15473 )
...
* Add preprocess_logits_for_metrics Trainer param
* Compute accuracy in LM examples
* Improve comments
2022-02-03 12:07:20 -05:00
Sylvain Gugger
45cac3fade
Fix labels stored in model config for token classification examples ( #15482 )
...
* Playing
* Properly set labels in model config for token classification example
* Port to run_ner_no_trainer
* Quality
2022-02-02 14:23:43 -05:00
Sylvain Gugger
d0b5ed110a
Harder check for IndexErrors in QA scripts ( #15438 )
...
* Harder check for IndexErrors in QA scripts
* Make test stronger
2022-02-01 15:49:13 -05:00
François REMY
0094eba363
Fix additional DataTrainingArguments documentation ( #15408 )
...
(This is an editorial change only)
2022-01-31 07:45:11 -05:00
Sylvain Gugger
c98a6ac211
Use argument for preprocessing workers in run_summairzation ( #15394 )
2022-01-28 18:34:10 -05:00
Lysandre
eab338104d
Docs for version v4.16.0
2022-01-27 13:11:51 -05:00
Lysandre
f87db5e412
Release: v4.16.0
2022-01-27 13:06:33 -05:00
François REMY
19732cc07a
Fix 'eval_split_name' described as defaulting to 'train' ( #15348 )
...
The default is correct (`test`) but the description is not.
2022-01-26 10:19:38 -05:00
Patrick von Platen
457dd4392b
[Examples] Correct run ner label2id for fine-tuned models ( #15017 )
...
* up
* up
* make style
* apply sylvains suggestions
* apply changes to accelerate as well
* more changes
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2022-01-24 21:18:04 +01:00
Sylvain Gugger
4cff3fae11
Second failing test
2022-01-21 12:19:28 -05:00
Sylvain Gugger
f6253147df
Skip failing test
2022-01-21 12:03:21 -05:00
NielsRogge
6c7b68d414
[ViTMAE] Add image pretraining script ( #15242 )
...
* Add script
* Improve script
* Fix data collator
* Update README
* Add label_names argument
* Apply suggestions from code review
* Add config parameters
* Update script
* Fix bug
* Improve README
* Improve README and add test
* Fix import
* Add image_column_name
2022-01-21 12:11:08 +01:00
Sylvain Gugger
531336bbfd
Fix deprecation warnings for int div ( #15180 )
...
* Fix deprecation warnings for int div
Co-authored-by: mgoldey <matthew.goldey@gmail.com>
* Fix import
* ensure that tensor output is python scalar
* make backward compatible
* make code more readable
* adapt test functions
Co-authored-by: mgoldey <matthew.goldey@gmail.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2022-01-18 07:28:53 -05:00
Sylvain Gugger
96881729ce
Remove assert on optional arg
2022-01-13 17:34:41 -05:00
Edoardo Federici
9a94bb8e21
mBART support for run_summarization.py ( #15125 )
...
* Update run_summarization.py
* Fixed languages and added missing code
* fixed obj, docs, removed source_lang and target_lang
* make style, run_summarization.py reformatted
2022-01-12 16:39:33 -05:00
Patrick von Platen
d72343d2b8
[Wav2Vec2 Speech Event] Add speech event v2 ( #15083 )
...
* up
* up
* up
* up
* up
* up
* improve
* up
* up
* Update src/transformers/trainer.py
* up
* up
* up
2022-01-10 10:46:21 +01:00
flozi00
b67f345d00
Update run_speech_recognition_seq2seq.py ( #14967 )
2022-01-06 19:26:45 +03:00
flozi00
774ed4a027
Fix Code block ( #14983 )
2022-01-04 12:59:20 +01:00
Patrick von Platen
600496fa50
[Wav2Vec2] Rename model's feature extractor to feature encoder ( #14959 )
...
* rename classes
* clean up more namings
* remove bogus file
* Apply suggestions from code review
* Apply suggestions from code review
* replace more names
* more regex replace
* make style
* correct
* correct more
* make style
* finish
* correct more in wav2vec2
* make style
* improve freeze_extractor
* add aliases
* add tf aliases
2021-12-28 20:33:23 +01:00
Patrick von Platen
f80775df2b
Update README.md ( #14965 )
2021-12-28 13:41:27 +01:00
Patrick von Platen
1c121916f3
Add Speech Seq2Seq Training script ( #14792 )
...
* start
* add gradient checkpointing and feature extractor freezing
* Apply suggestions from code review
* up
* up
* up
* correct
* up
* more changes
* up
* up
* up
* remove rst
2021-12-28 10:20:51 +01:00
Patrick von Platen
fa39ff9fc4
Docs for v4.16.0dev0
2021-12-22 20:39:44 +01:00
Patrick von Platen
05fa1a7ac1
Release: v4.15.0
2021-12-22 18:43:15 +01:00
Mario Šaško
1045a36c1f
Fix pytorch image classification example ( #14883 )
...
* Update example
* Remove skip in tests
2021-12-22 14:42:19 +01:00
Sylvain Gugger
e51c7b5872
Skip failing test
2021-12-21 15:15:17 -05:00
Stas Bekman
033c3ed95a
[examples/summarization] deal with None in data records ( #14816 )
...
* [examples/summarization] deal with None in data records
* rewrite to use a simpler (slower) variant
2021-12-21 09:17:28 -08:00
Patrick von Platen
7ae6f07004
[ASR example] Improve example + add more examples ( #14848 )
...
* up
* load up
* up
2021-12-21 13:12:22 +01:00
Patrick von Platen
c4a96cecbc
Wav2Vec2 meets phonemes ( #14353 )
...
* up
* add tokenizer
* improve more
* finish tokenizer
* finish
* adapt speech recognition script
* adapt convert
* more fixes
* more fixes
* update phonemizer wav2vec2
* better naming
* fix more tests
* more fixes swedish
* correct tests
* finish
* improve script
* remove file
* up
* lets get those 100 model architectures until the end of the month
* make fix-copies
* correct more
* correct script
* more fixes
* more fixes
* add to docs
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* replace assert
* fix copies
* fix docs
* new try docs
* boom boom
* update
* add phonemizer to audio tests
* make fix-copies
* up
* upload models
* some changes
* Update tests/test_tokenization_wav2vec2_phoneme.py
Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
* more fixes
* remove @
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
2021-12-17 19:56:44 +01:00
Lysandre
7c9c41f43c
Docs for v4.14.0
2021-12-15 18:29:53 +01:00
Lysandre
960d8cb41d
Release: v4.14.0
2021-12-15 18:20:35 +01:00
Josué Nascimento
971e36667a
Change how to load config of XLNetLMHeadModel ( #14746 )
2021-12-13 12:34:26 -05:00
Lysandre
ab31b3e41b
Docs for v4.14.0dev0
2021-12-09 17:09:23 +01:00
Lysandre
4da3a696e4
Release: v4.13.0
2021-12-09 16:55:21 +01:00
Gaurang Tandon
4ea19de80c
fix: verify jsonlines file in run_translation ( #14660 ) ( #14661 )
...
* fix: verify jsonl in run_translation (#14660 )
* fix(run_translation.py): json/jsonl validation
Both json and jsonl are to be accepted as valid jsonlines file extension
* fix(run_translation.py): make black happy
* Ran make style
2021-12-08 13:25:30 -05:00
Julien Chaumond
6cdc3a7844
[urls to hub] Replace outdated model tags with their now-canonical pipeline types ( #14617 )
...
* Replace outdated model tags with their now-canonical pipeline types
* spam the CI till it's green
2021-12-06 04:35:01 -05:00
Kamal Raj
803a8cd18f
updated readme with proper arguments ( #14624 )
2021-12-05 22:12:51 -05:00
(Bill) Yuchen Lin
3977b58437
fix a typo ( #14626 )
2021-12-05 11:31:23 +05:30
Nicholas Broad
69e16abf98
Switch from using sum for flattening lists of lists in group_texts ( #14472 )
...
* remove sum for list flattening
* change to chain(*)
* make chain object a list
* delete empty lines
per sgugger's suggestions
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Nicholas Broad <nicholas@nmbroad.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2021-11-22 16:17:26 -05:00
Stas Bekman
11f65d4158
[test] add test for --config_overrides ( #14466 )
...
* add test for --config_overrides
* remove unneeded parts of the test
2021-11-22 11:33:43 -05:00
Patrick von Platen
efea0f868b
[Speech Recognition] More examples
...
Add more XLS-R training runs to the official examples
2021-11-18 23:42:02 +01:00
William Held
01f8e639d3
Recover Deleted XNLI Instructions ( #14437 )
2021-11-17 20:16:47 -05:00
Patrick von Platen
55f49c5f4b
[Wav2Vec2 Example] Improve fine-tuning script ( #14373 )
...
* improve some stuff
* finish
* correct last
2021-11-12 16:35:57 +01:00
karthikrangasai
4f24058c58
Update Seq2Seq QA example script to use SQuAD metric. ( #14335 )
...
* Update postporcessing accordingly to use SQuAD metric.
* Update assets accordingly based on SQuAD metrics.
* Fix function naming error.
2021-11-09 08:04:23 -05:00
Sylvain Gugger
08a5f57567
Add new LFS prune API ( #14294 )
2021-11-05 18:58:51 -04:00
NielsRogge
7396095af7
Update README of QA examples ( #14172 )
2021-11-01 12:52:22 +01:00
Patrick von Platen
ba71f1b57f
Update README.md
2021-10-28 19:43:05 +02:00
Lysandre
b8fad022a0
v4.13.0.dev0
2021-10-28 12:56:46 -04:00
Lysandre
62bf536631
Release v4.12.0
2021-10-28 12:09:49 -04:00
Anton Lozhkov
78b6a2ecbd
Add audio-classification benchmarking results ( #14192 )
2021-10-28 15:59:18 +03:00
Patrick von Platen
88cd82e801
Update README.md
2021-10-28 02:35:01 +02:00
Patrick von Platen
e118db15d6
Update README.md
2021-10-28 01:59:27 +02:00
Patrick von Platen
01b1466983
[TPU tests] Enable first TPU examples pytorch ( #14121 )
...
* up
* up
* fix
* up
* Update examples/pytorch/test_xla_examples.py
* correct labels
* up
* up
* up
* up
* up
* up
2021-10-28 01:22:28 +02:00
Emanuel Huber
ebd48c6de5
Replace assertions with ValueError exception ( #14142 )
...
Updated masked-language modeling examples in pytorch
with convention defined by #12789
2021-10-26 17:14:29 -04:00
Matthew Goldey
42bfb83d74
fix typos in error messages in speech recognition example and modelcard.py ( #14166 )
...
* specify the text column name in the error message
* pluralize the word fields
2021-10-26 16:36:26 -04:00
Jangwon Park
41dad89f70
chore: typo on ner accelerate example code ( #14150 )
2021-10-26 16:23:41 -04:00
Patrick von Platen
9799f4e150
Update README.md
2021-10-26 18:59:25 +02:00
Patrick von Platen
f5ed19f57d
[Speech Recognition] - Distributed training: Make sure vocab file removal and creation don't interfer ( #14161 )
...
* up
* better
2021-10-26 15:59:33 +02:00
Patrick von Platen
e248e9b042
up ( #14154 )
2021-10-26 13:08:18 +02:00
Patrick von Platen
c99a2832ed
Update README.md
2021-10-25 19:50:36 +02:00
Patrick von Platen
1a9381c60d
Update README.md
2021-10-25 19:49:51 +02:00
karthikrangasai
1b871e091b
Supporting Seq2Seq model for question answering task ( #13432 )
...
* Add seq2seq example for QnA on SQuAD Dataset.
* Changes from review - Fixing styling mistakes.
* Added how to example in README, simplified the access to dataset's preprocess function.
* Added tests for the seq2seq QA example.
* Change dataset column name to fix tests.
* Fix test command mistake.
* Add missing argument 'ignore_pad_token_for_loss' from DataTrainingArguments.
* Add missing argument 'num_beams' from DataTrainingArguments.
* Fix processing of output predicted token ids so that tokenizer decode gets appropriate input. Updated assertion conditions on the tests.
2021-10-25 07:42:53 -04:00
lee1jun
d432a654f6
fix typo in license docstring ( #14094 )
...
last line: "# limitations under the License." is missing
2021-10-21 15:31:32 -04:00
Anton Lozhkov
e03544a138
[Examples] Add audio classification notebooks ( #14099 )
...
* Update SEW integration test tolerance
* Add audio classification notebooks
2021-10-21 19:15:46 +03:00
Patrick von Platen
e9d2a639f4
up ( #14093 )
2021-10-21 10:30:02 +02:00
Sylvain Gugger
f875fb0e5f
Fix label attribution in token classification examples ( #14055 )
2021-10-20 07:55:14 -04:00
Patrick von Platen
53dc39d821
up ( #14079 )
2021-10-20 13:01:42 +02:00
Patrick von Platen
0bc2e54f00
Add ASR colabs ( #14067 )
...
* up
* Update notebooks/README.md
2021-10-20 11:51:41 +02:00
Anton Lozhkov
dbaf49203e
[Examples] Use Audio feature in speech classification ( #14052 )
...
* Update SEW integration test tolerance
* Update audio classification
* Update test
* Remove torchaudio
* Add dataset revision
* Hub branch naming
* Revert dataset revisions
* Update datasets
2021-10-20 12:22:43 +03:00
Weizhe Yuan
7a3147e9b8
fix typo ( #14049 )
2021-10-18 18:03:11 -04:00
Patrick von Platen
bdf31d6e0a
[Speech] Move all examples to new audio feature ( #14045 )
...
* up
* up
* up
* finish
2021-10-18 12:52:40 +02:00
Patrick von Platen
37c5759cbe
[Speech Examples] Add new audio feature ( #14027 )
...
* finish
* up
* finish all
* up
2021-10-17 23:01:03 +02:00
Patrick von Platen
7fb2a8b3d9
up ( #14008 )
2021-10-14 15:46:22 +02:00
Sylvain Gugger
0ef61d392c
Revert "Skip faulty test"
...
This reverts commit 5b6bd4e788
2021-10-14 09:02:41 -04:00
Sylvain Gugger
5b6bd4e788
Skip faulty test
2021-10-13 22:04:40 -04:00
Patrick von Platen
d45fc7da3d
[Speech Examples] Add pytorch speech pretraining ( #13877 )
...
* adapt wav2vec2
* add example
* add files
* adapt
* remove bogus file
* Apply suggestions from code review
* adapt files more
* upload changes
* del old files
* up
* up
* up
* up
* up
* correct gradient checkpoitning
* add readme
* finish
* finish
* up
* more fixes
* up
* up
* add demo run to readme
* up
2021-10-12 00:46:32 +02:00
Chungman Lee
46dfe99e44
Fix typo in README.md ( #13883 )
2021-10-08 14:25:32 -04:00
Dhananjay Shettigar
319beb64eb
#12789 Replace assert statements with exceptions ( #13909 )
...
* #12789 Replace assert statements with exceptions
* fix-copies: made copy changes to utils_qa.py in examples/pytorch/question-answering and examples/tensorflow/question-answering
* minor refactor for clarity
2021-10-07 09:09:01 -04:00
Akul Agrawal
dac7798144
Update run_qa.py ( #13857 )
2021-10-05 23:10:24 -04:00
Nathan Raw
cc0a415e2f
✨ update image classification example ( #13824 )
...
* ✨ update image classification example
* 📌 update reqs
2021-10-04 11:49:51 -07:00
Anton Lozhkov
4213728067
[Examples] Add an official audio classification example ( #13722 )
...
* Restore broken merge
* Additional args, DDP, remove CommonLanguage
* Update examples for V100, add training results
* Style
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Remove custom datasets for simplicity, apply suggestions from code review
* Add the attention_mask flag, reorganize README
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2021-10-01 18:52:45 +02:00
Patrick von Platen
44eb8bdeea
map only on one process ( #13810 )
2021-09-30 18:52:53 +02:00
Stas Bekman
b90096fe14
[examples run_glue.py] missing requirements scipy, sklearn ( #13768 )
...
* missing requirement
* list both
2021-09-29 13:45:19 -07:00
Lysandre
11c69b8045
Docs for version v4.11.0
2021-09-27 14:19:38 -04:00
Lysandre
dc193c906d
Release: v4.11.0
2021-09-27 14:14:09 -04:00
Sylvain Gugger
044eff5bf0
Update requirements for speech example ( #13745 )
2021-09-26 09:02:45 +02:00
Patrick von Platen
469b80d4e7
Update README.md
2021-09-24 18:53:58 +02:00
Patrick von Platen
493643fff8
up ( #13733 )
2021-09-24 18:32:35 +02:00
Gunjan Chhablani
38580455de
Add model card creation snippet to example scripts ( #13730 )
...
* Update run_glue.py
* Update run_glue.py
* Add model creation snippet to other scripts
* Fix style
2021-09-24 15:51:46 +02:00
Patrick von Platen
95f888fd6a
Update README.md
2021-09-24 09:53:37 +02:00
Patrick von Platen
4a320f6c9a
[ASR] Add official ASR CTC example to examples/pytorch/speech-recognition ( #13620 )
...
* up
* rename
* add asr example
* add auto feature extractor
* some more fixes
* correct layerdrop
* correct for multi-gpu dist
* clean up
* refactor
* refactor
* more fixes
* more fixes
* clean-up
* finish
* up
* Apply suggestions from code review
* fix isort
* update
* up
* add note
* apply surajs suggestions
* Apply suggestions from code review
Co-authored-by: Suraj Patil <surajp815@gmail.com>
* isort
* small change
* Apply suggestions from code review
Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
* Apply suggestions from code review
Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
* add hubert
* Update examples/pytorch/speech-recognition/run_speech_recognition_ctc.py
Co-authored-by: Suraj Patil <surajp815@gmail.com>
Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
2021-09-24 07:01:11 +02:00
Sylvain Gugger
27d4639779
Make gradient_checkpointing a training argument ( #13657 )
...
* Make gradient_checkpointing a training argument
* Update src/transformers/modeling_utils.py
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
* Update src/transformers/configuration_utils.py
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
* Fix tests
* Style
* document Gradient Checkpointing as a performance feature
* Small rename
* PoC for not using the config
* Adapt BC to new PoC
* Forgot to save
* Rollout changes to all other models
* Fix typo
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
Co-authored-by: Stas Bekman <stas@stason.org>
2021-09-22 07:51:38 -04:00
Sylvain Gugger
b7d264be0d
Add push_to_hub to no_trainer examples ( #13659 )
...
* Add push_to_hub to no_trainer examples
* Quality
* Document integration
* Roll out to other examples
2021-09-21 13:13:30 -04:00
Suraj Patil
87d5057d86
fix typo ( #13647 )
2021-09-20 13:22:26 +05:30
Patrick von Platen
95f933ea85
[Pretrained Model] Add resize_position_embeddings ( #13559 )
...
* finish
* delete bogus file
* correct some stuff
* finish
* finish
2021-09-15 19:03:56 +02:00
Aleksander Smywiński-Pohl
008c2d0b7a
Fix typo in documentation ( #13494 )
...
* Fix typo in deepspeed documentation
* Add missing import in deepspeed configuration
* Fix path in translation examples
2021-09-09 08:00:05 -04:00
Nathan Raw
79815090ea
Fix img classification tests ( #13456 )
...
* ✅ Update image-classification example's tests
* 🔥 remove cats_and_dogs test samples
* 💄 fix flake8
2021-09-07 05:58:45 -04:00
Suraj Patil
2dd975b235
skip image classification test ( #13451 )
2021-09-06 21:46:25 +05:30
Suraj Patil
6b29bff852
add torchvision in example test requirements ( #13438 )
2021-09-06 15:17:54 +02:00
Nathan Raw
76c4d8bf26
✨ Add PyTorch image classification example ( #13134 )
...
* ✨ add pytorch image classification example
* 🔥 remove utils.py
* 💄 fix flake8 style issues
* 🔥 remove unnecessary line
* ✨ limit dataset sizes
* 📌 update reqs
* 🎨 restructure - use datasets lib
* 🎨 import transforms directly
* 📝 add comments
* 💄 style
* 🔥 remove flag
* 📌 update requirement warning
* 📝 add vision README.md
* 📝 update README.md
* 📝 update README.md
* 🎨 add image-classification tag to model card
* 🚚 rename vision ➡️ image-classification
* 📝 update image-classification README.md
2021-09-02 13:29:42 -06:00
Lysandre
5ee67a4412
Docs for v4.10.0
2021-08-31 16:02:31 +02:00
Lysandre
d12bbe4942
Release: v4.10.0
2021-08-31 15:53:10 +02:00
Sylvain Gugger
c76de1053e
Add generate kwargs to Seq2SeqTrainingArguments ( #13339 )
...
* Add generate kwargs to Seq2SeqTrainingArguments
* typo
* Address review comments + doc
* Style
2021-08-31 08:42:00 -04:00
Sylvain Gugger
139e830158
Update label2id in the model config for run_glue ( #13334 )
2021-08-30 10:35:09 -04:00
Stefan Schweter
4046e66e40
examples: only use keep_linebreaks when reading TXT files ( #13320 )
...
* examples: only use keep_linebreaks when reading TXT files for all CLM examples
* examples: only use keep_linebreaks when reading TXT files for all CLM examples
* examples: only use keep_linebreaks when reading TXT files for all CLM examples
2021-08-28 16:22:29 +02:00
Stefan Schweter
319d840b46
examples: add keep_linebreaks option to CLM examples ( #13150 )
...
* examples: add keep_linebreaks option to text dataset loader for all CLM examples
* examples: introduce new keep_linebreaks option as data argument in CLM examples
2021-08-27 11:35:45 +02:00
Allan Lin
91ff480e26
Update namespaces inside torch.utils.data to the latest. ( #13167 )
...
* Update torch.utils.data namespaces to the latest.
* Format
* Update Dataloader.
* Style
2021-08-19 14:29:51 +02:00
Sylvain Gugger
7fcee113c1
Tpu tie weights ( #13030 )
...
* Fix tied weights on TPU
* Manually tie weights in no trainer examples
* Fix for test
* One last missing
* Gettning owned by my scripts
* Address review comments
* Fix test
* Fix tests
* Fix reformer tests
2021-08-06 20:41:39 +02:00
Chungman Lee
75b8990d90
fix typo in example/text-classification README ( #12974 )
...
* fix typo in example/text-classification README
* add space to align the table
2021-08-02 12:58:43 +02:00
Sylvain Gugger
3ec851dc5e
Fix QA examples for roberta tokenizer ( #12928 )
2021-07-28 09:47:49 -04:00
Sylvain Gugger
fd85734e0e
Add option to set max_len in run_ner ( #12929 )
2021-07-28 09:38:12 -04:00
Sylvain Gugger
303989de0e
Add accelerate to examples requirements ( #12888 )
2021-07-26 09:57:34 -04:00
Lysandre
40de2d5a4f
Docs for v4.10.0dev0
2021-07-22 12:52:25 +02:00
Lysandre
72aee83ced
Release: v4.9.0
2021-07-22 12:11:55 +02:00
Maxwell Forbes
fcf83011df
Fix type of max_seq_length arg in run_swag.py ( #12832 )
2021-07-22 02:14:14 -04:00
Sylvain Gugger
6f1adc4334
Fix group_lengths for short datasets ( #12558 )
2021-07-08 07:23:41 -04:00
Souvic Chakraborty
1d6623c6a2
MLM training fails with no validation file(same as #12406 for pytorch now) ( #12517 )
...
* Validation split percentage to be used for custom data files also
Issue same as https://github.com/huggingface/transformers/issues/12406 fixed for pytorch branch run_mlm.py
* Validation split added in the right place
* Update run_clm.py
* validation split added for custom files
* Validation split added for custom files
* Update run_plm.py
* fixed validation split for custom files as input for pytorch examples in lm
* Update run_clm_no_trainer.py
* args modified
2021-07-07 09:05:44 -04:00
Bhadresh Savani
04dbea31a9
[Examples] Added context manager to datasets map ( #12367 )
...
* added cotext manager to datasets map
* fixed style and spaces
* fixed warning of deprecation
* changed desc
2021-06-28 09:14:00 -07:00
Taha ValizadehAslani
9490d668d2
Update run_mlm.py ( #12344 )
...
Before the code could not be used for validation only because of this line:
extension = data_args.train_file.split(".")[-1]
was assuming that extension must be extracted from the training dataset. This line would run regardless of the training or validation options of the user. This would lead to an error if the user only wants to run an evaluation only and does not want to do train (because the training file does not exist). I modified it to extract extension from the training file if the user wants to do train and extract it from the validation file if the user wants to run eval. This way the code can be used for both training and validation separately.
2021-06-28 07:49:22 -04:00
Bhadresh Savani
ff5cdc086b
replace print with logger ( #12368 )
2021-06-26 09:31:25 -07:00
Bhadresh Savani
539ee456d4
[Examples] Replicates the new --log_level feature to all trainer-based pytorch ( #12359 )
...
* added log_level
* fix comment
* fixed log_level
* Trigger CI
* Unfied logging
* simplified args for log_level
2021-06-25 14:58:42 -07:00
Stas Bekman
64e6098094
[trainer] add main_process_first context manager ( #12351 )
...
* main_process_first context manager
* handle multi-node, add context description
* sync desc
2021-06-25 14:58:03 -07:00
Stas Bekman
4a872caef4
remove extra white space from log format ( #12360 )
2021-06-25 13:20:14 -07:00
michal pitr
d4ce31e839
fixed typo ( #12356 )
2021-06-25 07:49:29 -04:00
Sylvain Gugger
2150dfed31
v4.9.0.dev0
2021-06-23 13:31:19 -04:00
Sylvain Gugger
9252a5127f
Release: v4.8.0
2021-06-23 13:25:56 -04:00
Stas Bekman
ebe5413589
[trainer] 2 bug fixes and a rename ( #12309 )
...
* bug fixes and a rename
* add extended DDP test
2021-06-22 11:13:23 -07:00
Stas Bekman
dad414d5f9
[trainer + examples] set log level from CLI ( #12276 )
...
* set log level from CLI
* add log_level_replica + test + extended docs
* cleanup
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* rename datasets objects to allow datasets module
* improve the doc
* style
* doc improve
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2021-06-21 19:30:50 -07:00
Bhavitvya Malik
e43e11260f
update desc for map in all examples ( #12226 )
...
* update desc for map in all examples
* added plm
* suggestions
2021-06-17 15:37:31 -04:00
Lysandre
0daadc1919
Docs for v4.8.0
2021-06-17 18:17:42 +02:00
Lysandre
7a6c9fab8e
Release: v4.7.0
2021-06-17 17:57:42 +02:00
Sylvain Gugger
7d7ceca396
Model card defaults ( #12122 )
...
* [WIP] Model card defaults
* finetuned_from default value
* Add all mappings to the mapping file
* Be more defensive on finetuned_from arg
* Add default task tag
* Separate tags from tasks
* Edge case for dataset
* Apply suggestions from code review
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
2021-06-15 16:01:37 -04:00
kumapo
955b2b97a6
Enable add_prefix_space if model_type is roberta or gpt2 ( #12116 )
2021-06-15 09:33:21 -04:00
Kumar Abhishek
9de62cfbce
[lm examples] Replicate --config_overrides addition to other LM examples ( #12135 )
...
* [lm examples] Replicate --config_overrides addition to other LM examples
* Removing no trainer files changes
* Update README
Co-authored-by: Kumar Abhishek <kabhishek@expedia.com>
2021-06-14 08:12:22 -04:00
Nicholas Broad
cd7961b632
Use text_column_name variable instead of "text" ( #12132 )
...
* Use text_column_name variable instead of "text"
`text_column_name` was already defined above where I made the changes and it was also used below where I made changes.
This is a very minor change. If a dataset does not use "text" as the column name, then the `tokenize_function` will now use whatever column is assigned to `text_column_name`. `text_column_name` is just the first column name if "text" is not a column name. It makes the function a little more robust, though I would assume that 90% + of datasets use "text" anyway.
* black formatting
* make style
Co-authored-by: Nicholas Broad <nicholas@nmbroad.com>
2021-06-14 08:11:13 -04:00
Sylvain Gugger
b8ab541340
Don't log anything before logging is setup in examples ( #12121 )
...
* Don't log anything before logging is setup in examples
* Last example
2021-06-14 08:03:33 -04:00
Bhavitvya Malik
d2753dcbec
add relevant description to tqdm in examples ( #11927 )
...
* add relevant `desc` in examples
* require_version datasets>=1.8.0
2021-06-10 15:59:55 -04:00
Matt
bebbdd0fc9
Appending label2id and id2label to models to ensure inference works properly ( #12102 )
2021-06-10 15:25:04 +01:00
Sylvain Gugger
d72e5a3a6d
Fix quality
2021-06-10 09:27:11 -04:00
kumapo
472a867626
Add text_column_name and label_column_name to run_ner and run_ner_no_trainer args ( #12083 )
...
* Add text_column_name and label_column_name to run_ner args
* Minor fix: grouping for text and label column name
2021-06-10 08:03:20 -04:00
Koichi Yasuoka
82a2b76c95
Update run_ner.py with id2label config ( #12001 )
2021-06-09 07:27:05 -04:00
Sylvain Gugger
fd6902838a
Properly indent block_size ( #12070 )
2021-06-08 10:27:02 -04:00
cdleong
49bee0aea4
Add torch to requirements.txt in language-modeling ( #12040 )
...
* Add torch to requirements.txt in language-modeling
* Update examples/pytorch/language-modeling/requirements.txt
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2021-06-08 09:02:35 -04:00
Russell Klopfer
e363e1d936
adds metric prefix. ( #12057 )
...
* adds metric prefix.
* update tests to include prefix
2021-06-07 22:34:10 -04:00
Fan Zhang
7e73601f32
modify qa-trainer ( #11872 )
...
* modify qa-trainer
* fix flax model
2021-06-01 08:28:41 -04:00
Philip May
cfca638acb
Add MT5ForConditionalGeneration as supported arch. to summarization README ( #11961 )
...
* Add MT5ForConditionalGeneration as supported arch.
* Update README.md
2021-05-31 21:24:33 +05:30
Stas Bekman
1b6530104d
[Examples] create model with custom config on the fly ( #11798 )
...
* create custom model on the flight
* better wording
* add update_from_string
* cleanup
* cleanup
* Update src/transformers/configuration_utils.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* more bool options
* style
* fix logger
* add test
* add the doc
* assert on conflict of options
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2021-05-25 10:40:49 -07:00
Stas Bekman
6287c929c1
[lm examples] fix overflow in perplexity calc ( #11855 )
...
* fix overflow in perplexity calc
* use inf
* fix
2021-05-25 08:11:26 -07:00
Sylvain Gugger
f086652b16
Add option to log only once in multinode training ( #11819 )
...
* Add option to long only once in multinode training
* Use an alternate property
2021-05-25 08:03:43 -04:00
Wang Ran (汪然)
b8344a274f
typo ( #11858 )
2021-05-25 04:23:46 -04:00
Keren Fuentes
223943872e
Fix failing test on Windows Platform ( #11589 )
...
* add separator for windows
* fixes test_is_copy_consistent on Windows
* fixing writing encoding issue on extended test (for Windows)
* resolving comments
2021-05-20 19:54:23 -04:00
Tomy Hsieh
eb3e072a3b
Fix a small error in summarization example ( #11762 )
2021-05-18 14:38:36 -04:00
Philipp Schmid
04e25c6286
add dataset_name to data_args and added accuracy metric ( #11760 )
...
* add `dataset_name` to data_args and added accuracy metric
* added documentation for dataset_name
* spelling correction
2021-05-18 16:27:29 +02:00
Patrick von Platen
cebb96f53a
Add more subsections to main doc ( #11758 )
...
* add headers to main doc
* Apply suggestions from code review
* update
* upload
2021-05-18 14:38:56 +01:00
Tommy Chiang
da7e73b721
Fix incorrect newline in #11650 ( #11757 )
2021-05-18 15:28:13 +02:00
Sylvain Gugger
936b57158a
Use new evaluation loop in TrainerQA ( #11746 )
2021-05-17 10:10:13 -04:00
Lysandre
d77eb0cf92
Docs for v4.7.0.dev0
2021-05-12 17:08:35 +02:00
Lysandre
64e78564a5
Release: v4.6.0
2021-05-12 17:03:03 +02:00
Sylvain Gugger
a135f59536
Auto modelcard ( #11599 )
...
* Autogenerate model cards from the Trainer
* ModelCard deprecated
* Fix test
* Style
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* Address review comments
* Quality
* With all metadata
* Metadata
* Post-merge conflict mess
* Data args and all examples
* Default license and languages when possible
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2021-05-11 11:30:34 -04:00
Jonathan Chang
64232bc0df
Add --text_column to run_summarization_no_trainer ( #11673 )
2021-05-11 07:58:38 -04:00
Tommy Chiang
7e406f4a65
[Examples] Fix invalid links after reorg ( #11650 )
2021-05-10 11:16:48 +05:30
Tommy Chiang
f2ffcaf49f
[Examples] Check key exists in datasets first ( #11503 )
2021-05-09 15:42:38 -04:00
Stas Bekman
ba0d50f214
[examples] fix sys.path in conftest.py ( #11636 )
...
* restore conftest.py
* fix conftest and make copies
* remove unneeded parts
* remove unwanted files
2021-05-07 14:44:22 -07:00
Jonathan Chang
6f40e31766
Fix comment in run_clm_no_trainer.py ( #11624 )
2021-05-07 12:32:30 +05:30
Vipul Raheja
f594090a93
fix typo in command ( #11605 )
2021-05-06 12:32:54 +05:30
Patrick von Platen
3e3e41ae20
Pytorch - Lazy initialization of models ( #11471 )
...
* lazy_init_weights
* remove ipdb
* save int
* add necessary code
* remove unnecessary utils
* Update src/transformers/models/t5/modeling_t5.py
* clean
* add tests
* correct
* finish tests
* finish tests
* fix some more tests
* fix xlnet & transfo-xl
* fix more tests
* make sure tests are independent
* fix tests more
* finist tests
* final touches
* Update src/transformers/modeling_utils.py
* Apply suggestions from code review
* Update src/transformers/modeling_utils.py
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
* Update src/transformers/modeling_utils.py
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
* clean tests
* give arg positive name
* add more mock weights to xlnet
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
2021-05-05 17:22:20 +02:00
Sylvain Gugger
6b241e0e3b
Reproducible checkpoint ( #11582 )
...
* Set generator in dataloader
* Use generator in all random samplers
* Checkpoint all RNG states
* Final version
* Quality
* Test
* Address review comments
* Quality
* Remove debug util
* Add python and numpy RNGs
* Split states in different files in distributed
* Quality
* local_rank for TPUs
* Only use generator when accepted
* Add test
* Set seed to avoid flakiness
* Make test less flaky
* Quality
2021-05-04 16:20:56 -04:00
Sylvain Gugger
87dd1a00ef
Fix metric computation in run_glue_no_trainer ( #11569 )
2021-05-03 11:42:55 -04:00
Bhadresh Savani
84326a28f8
[Examples] Added support for test-file in QA examples with no trainer ( #11510 )
...
* added support for test-file
* fixed typo
* added suggested changes
* reformatted code
* modifed files
* fix post processing error
* Trigger CI
* removed extra lines
2021-04-30 09:02:50 -04:00
Suraj Patil
57c8e822f7
reszie token embeds ( #11524 )
2021-04-30 08:47:01 -04:00
Sylvain Gugger
b29eb247d3
Split checkpoint from model_name_or_path in examples ( #11492 )
...
* Split checkpoint from model_name_or_path in examples
* Address review comments
* Address review comments
2021-04-29 18:33:47 -04:00
Bhadresh Savani
1d30ec95c7
[Examples] Fixes inconsistency around eval vs val and predict vs test ( #11380 )
...
* added changes for uniformity
* modified files
* corrected typo
* fixed qa scripts
* fix typos
* fixed predict typo in qa no trainer
* fixed test file
* reverted trainer changes
* reverted trainer changes in custom exmaples
* updated readme
* added changes in deepspeed test
* added changes for predict and eval
2021-04-26 09:24:31 -07:00
Amine Abdaoui
e3e70f9551
docs(examples): fix link to TPU launcher script ( #11427 )
2021-04-26 09:08:43 -04:00
Sylvain Gugger
1ef152eb48
Default to accuracy metric ( #11405 )
2021-04-23 14:49:59 -04:00
Sylvain Gugger
bf2e0cf70b
Trainer push to hub ( #11328 )
...
* Initial support for upload to hub
* push -> upload
* Fixes + examples
* Fix torchhub test
* Torchhub test I hate you
* push_model_to_hub -> push_to_hub
* Apply mixin to other pretrained models
* Remove ABC inheritance
* Add tests
* Typo
* Run tests
* Install git-lfs
* Change approach
* Add push_to_hub to all
* Staging test suite
* Typo
* Maybe like this?
* More deps
* Cache
* Adapt name
* Quality
* MOAR tests
* Put it in testing_utils
* Docs + torchhub last hope
* Styling
* Wrong method
* Typos
* Update src/transformers/file_utils.py
Co-authored-by: Julien Chaumond <julien@huggingface.co>
* Address review comments
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Julien Chaumond <julien@huggingface.co>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2021-04-23 09:17:37 -04:00
Yoshitomo Matsubara
c3d6f33918
fixed typos ( #11391 )
2021-04-23 07:48:42 -04:00
Max Del
a90d3f1862
Fix typo in text ( #11396 )
2021-04-23 07:37:19 -04:00
johnson7788
5b5e4ca366
[run_translation.py] fix typo ( #11372 )
...
fix typo
Co-authored-by: johnson <johnson@github.com>
2021-04-22 17:47:11 +05:30
Sylvain Gugger
dabeb15292
Examples reorg ( #11350 )
...
* Base move
* Examples reorganization
* Update references
* Put back test data
* Move conftest
* More fixes
* Move test data to test fixtures
* Update path
* Apply suggestions from code review
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
* Address review comments and clean
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
2021-04-21 11:11:20 -04:00
