Zachary Mueller
d57da99237
Add tests for no_trainer and fix existing examples ( #16656 )
...
* Fixed some bugs involving saving during epochs
* Added tests mimicking the existing examples tests
* Added in json exporting to all `no_trainer` examples for consistency
2022-04-08 10:03:56 -04:00
Zachary Mueller
febe42b5da
Update no_trainer scripts with new Accelerate functionalities ( #16617 )
...
Adds logging and save/loading to the Accelerate scripts
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2022-04-06 15:29:32 -04:00
Lysandre Debut
a180efe7fd
Dev version
2022-04-06 11:08:12 -04:00
Karim Foda
24a85cca61
Add use_auth to load_datasets for private datasets to PT and TF examples ( #16521 )
...
* fix formatting and remove use_auth
* Add use_auth_token to Flax examples
2022-04-04 10:27:45 -04:00
Bhadresh Savani
05b4c32908
fixed a typo ( #16508 )
2022-03-31 07:49:02 -04:00
Stas Bekman
a73281e3e4
[examples] max samples can't be bigger than the len of dataset ( #16501 )
...
* [examples] max samples can't be bigger than then len of dataset
* do tf and flax
2022-03-30 12:33:16 -07:00
Sylvain Gugger
b62ac4d240
Fix example test and test_fetcher for examples ( #16478 )
2022-03-29 12:21:19 -04:00
Eldar Kurtic
5216607f8a
[MNLI example] Prevent overwriting matched with mismatched metrics ( #16475 )
...
* Prevent overwriting matched with mismatched metrics
* Fix style
2022-03-29 10:38:14 -04:00
Sylvain Gugger
867f3950fa
Rename master to main for notebooks links and leftovers ( #16397 )
2022-03-25 09:12:23 -04:00
Sylvain Gugger
088c1880b7
Big file_utils cleanup ( #16396 )
...
* Big file_utils cleanup
* This one still needs to be treated separately
2022-03-25 07:25:20 -04:00
Sylvain Gugger
4975002df5
Reorganize file utils ( #16264 )
...
* Split file_utils in several submodules
* Fixes
* Add back more objects
* More fixes
* Who exactly decided to import that from there?
* Second suggestion to code with code review
* Revert wront move
* Fix imports
* Adapt all imports
* Adapt all imports everywhere
* Revert this import, will fix in a separate commit
2022-03-23 10:26:33 -04:00
Lysandre Debut
eca77f4719
Updates the default branch from master to main ( #16326 )
...
* Updates the default branch from master to main
* Links from `master` to `main`
* Typo
* Update examples/flax/README.md
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2022-03-23 03:46:59 -04:00
Sylvain Gugger
19597998f6
Don't compute metrics in LM examples on TPU ( #16029 )
2022-03-10 07:44:51 -05:00
Shotaro Ishihara
8feede229c
Fix broken code blocks in README.md ( #15967 )
...
at transformers/examples/pytorch/contrastive-image-text
2022-03-09 17:07:52 +01:00
Joao Gante
e7f34ccd4f
Swag example: Update doc format ( #16014 )
2022-03-09 13:25:34 +00:00
davidleonfdez
c0281feb50
Fix #15898 ( #15928 )
2022-03-03 14:41:03 -05:00
Sylvain Gugger
79d28e80b6
v4.18.0.dev.0
2022-03-03 10:19:58 -05:00
Suraj Patil
bf1fe32824
[examples/summarization and translation] fix readme ( #15833 )
2022-02-25 17:28:16 +01:00
Lysandre Debut
29c10a41d0
[Test refactor 1/5] Per-folder tests reorganization ( #15725 )
...
* Per-folder tests reorganization
Co-authored-by: sgugger <sylvain.gugger@gmail.com>
Co-authored-by: Stas Bekman <stas@stason.org>
2022-02-23 15:46:28 -05:00
Yongrae Jo
3db2e8f92b
Fix typo on examples/pytorch/question-answering ( #15644 )
...
cna -> can
2022-02-22 13:51:07 -05:00
Joao Gante
3956b133b6
TF text classification examples ( #15704 )
...
* Working example with to_tf_dataset
* updated text_classification
* more comments
2022-02-21 17:17:59 +00:00
Suraj Patil
86119c1154
add VisionTextDualEncoder and CLIP fine-tuning script ( #15701 )
...
* begin script
* update script
* fix features and data args
* main
* add requirements
* add column name args
* fix captions
* don't jit transforms
* fix caption
* fix labels, handle attention mask
* convert pixel values to numpy
* labels => input_ids
* transform images on the fly
* use AutoModel class, create the hybird model outside of the script
* fix version message
* add readme
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* adderss review comments
* add more comments
* allow freezing vision and text models
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2022-02-21 16:10:59 +01:00
Simon Sardorf
a63bd3675f
Remove input and target reset after preprocessing ( #15741 )
...
Remove input and target reset after preprocessing
2022-02-21 11:10:15 +01:00
NielsRogge
57882177be
Add SimMIM ( #15586 )
...
* Add first draft
* Make model importable
* Make SwinForMaskedImageModeling importable
* Fix imports
* Add missing inits
* Add support for Swin
* Fix bug
* Fix bug
* Fix another bug
* Fix Swin MIM implementation
* Fix default encoder stride
* Fix Swin
* Add print statements for debugging
* Add image_size data argument
* Fix Swin
* Fix image_size
* Add print statements for debugging
* Fix print statement
* Remove print statements
* Improve reshaping of bool_masked_pos
* Add support for DeiT, fix tests
* Improve docstrings
* Apply new black version
* Improve script
* Fix bug
* Improve README
* Apply suggestions from code review
* Remove DS_Store and add to gitignore
* Apply suggestions from code review + fix BEiT Flax
* Revert BEiT changes
* Improve README
* Fix code quality
* Improve README
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MBP.localdomain>
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
2022-02-17 19:44:55 +01:00
NielsRogge
0e91f885c3
Add image classification notebook ( #15667 )
...
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
2022-02-17 13:14:01 +01:00
Patrick von Platen
3d5dea9bf0
Add example batch size to all commands ( #15596 )
2022-02-10 08:52:07 -05:00
Lysandre Debut
7732d0fe7a
Upgrade black to version ~=22.0 ( #15565 )
...
* Upgrade black to version ~=22.0
* Check copies
* Fix code
2022-02-09 09:28:57 -05:00
Anton Lozhkov
a459f7f97d
Add ASR CTC streaming example ( #15309 )
...
* Single-epoch run
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* Infinite dataset
* Trainer fix + distributed benchmark
* Benchmark fix
* unused import
* interleaved splits
* interleaved splits
* has_length util
* Move to research projects
* Leftover Sized checks
* Bump min version
* Unused import
* Revert trainer changes
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2022-02-07 18:35:37 +03:00
davidleonfdez
f1a4c4ead5
[WIP] Add preprocess_logits_for_metrics Trainer param ( #15473 )
...
* Add preprocess_logits_for_metrics Trainer param
* Compute accuracy in LM examples
* Improve comments
2022-02-03 12:07:20 -05:00
Sylvain Gugger
45cac3fade
Fix labels stored in model config for token classification examples ( #15482 )
...
* Playing
* Properly set labels in model config for token classification example
* Port to run_ner_no_trainer
* Quality
2022-02-02 14:23:43 -05:00
Sylvain Gugger
d0b5ed110a
Harder check for IndexErrors in QA scripts ( #15438 )
...
* Harder check for IndexErrors in QA scripts
* Make test stronger
2022-02-01 15:49:13 -05:00
François REMY
0094eba363
Fix additional DataTrainingArguments documentation ( #15408 )
...
(This is an editorial change only)
2022-01-31 07:45:11 -05:00
Sylvain Gugger
c98a6ac211
Use argument for preprocessing workers in run_summairzation ( #15394 )
2022-01-28 18:34:10 -05:00
Lysandre
eab338104d
Docs for version v4.16.0
2022-01-27 13:11:51 -05:00
Lysandre
f87db5e412
Release: v4.16.0
2022-01-27 13:06:33 -05:00
François REMY
19732cc07a
Fix 'eval_split_name' described as defaulting to 'train' ( #15348 )
...
The default is correct (`test`) but the description is not.
2022-01-26 10:19:38 -05:00
Patrick von Platen
457dd4392b
[Examples] Correct run ner label2id for fine-tuned models ( #15017 )
...
* up
* up
* make style
* apply sylvains suggestions
* apply changes to accelerate as well
* more changes
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2022-01-24 21:18:04 +01:00
Sylvain Gugger
4cff3fae11
Second failing test
2022-01-21 12:19:28 -05:00
Sylvain Gugger
f6253147df
Skip failing test
2022-01-21 12:03:21 -05:00
NielsRogge
6c7b68d414
[ViTMAE] Add image pretraining script ( #15242 )
...
* Add script
* Improve script
* Fix data collator
* Update README
* Add label_names argument
* Apply suggestions from code review
* Add config parameters
* Update script
* Fix bug
* Improve README
* Improve README and add test
* Fix import
* Add image_column_name
2022-01-21 12:11:08 +01:00
Sylvain Gugger
531336bbfd
Fix deprecation warnings for int div ( #15180 )
...
* Fix deprecation warnings for int div
Co-authored-by: mgoldey <matthew.goldey@gmail.com>
* Fix import
* ensure that tensor output is python scalar
* make backward compatible
* make code more readable
* adapt test functions
Co-authored-by: mgoldey <matthew.goldey@gmail.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2022-01-18 07:28:53 -05:00
Sylvain Gugger
96881729ce
Remove assert on optional arg
2022-01-13 17:34:41 -05:00
Edoardo Federici
9a94bb8e21
mBART support for run_summarization.py ( #15125 )
...
* Update run_summarization.py
* Fixed languages and added missing code
* fixed obj, docs, removed source_lang and target_lang
* make style, run_summarization.py reformatted
2022-01-12 16:39:33 -05:00
Patrick von Platen
d72343d2b8
[Wav2Vec2 Speech Event] Add speech event v2 ( #15083 )
...
* up
* up
* up
* up
* up
* up
* improve
* up
* up
* Update src/transformers/trainer.py
* up
* up
* up
2022-01-10 10:46:21 +01:00
flozi00
b67f345d00
Update run_speech_recognition_seq2seq.py ( #14967 )
2022-01-06 19:26:45 +03:00
flozi00
774ed4a027
Fix Code block ( #14983 )
2022-01-04 12:59:20 +01:00
Patrick von Platen
600496fa50
[Wav2Vec2] Rename model's feature extractor to feature encoder ( #14959 )
...
* rename classes
* clean up more namings
* remove bogus file
* Apply suggestions from code review
* Apply suggestions from code review
* replace more names
* more regex replace
* make style
* correct
* correct more
* make style
* finish
* correct more in wav2vec2
* make style
* improve freeze_extractor
* add aliases
* add tf aliases
2021-12-28 20:33:23 +01:00
Patrick von Platen
f80775df2b
Update README.md ( #14965 )
2021-12-28 13:41:27 +01:00
Patrick von Platen
1c121916f3
Add Speech Seq2Seq Training script ( #14792 )
...
* start
* add gradient checkpointing and feature extractor freezing
* Apply suggestions from code review
* up
* up
* up
* correct
* up
* more changes
* up
* up
* up
* remove rst
2021-12-28 10:20:51 +01:00
Patrick von Platen
fa39ff9fc4
Docs for v4.16.0dev0
2021-12-22 20:39:44 +01:00
