riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-21 13:38:31 +06:00
Code Issues Actions 7 Packages Projects Releases Wiki Activity
13,875 Commits 876 Branches 227 Tags 2 GiB
3587769c08
Commit Graph

125 Commits

Author SHA1 Message Date
Sylvain Gugger
5c67682b16
v4.33.0.dev0 2023-08-21 07:07:04 -04:00
Zach Mueller
01ab39b65f
Load state in else (#25318) 
* Load else

* New approach

* Propagate
 2023-08-08 05:41:00 -04:00
Jackmin801
145109382a
Allow trust_remote_code in example scripts (#25248) 
* pytorch examples

* pytorch mim no trainer

* cookiecutter

* flax examples

* missed line in pytorch run_glue

* tensorflow examples

* tensorflow run_clip

* tensorflow run_mlm

* tensorflow run_ner

* tensorflow run_clm

* pytorch example from_configs

* pytorch no trainer examples

* Revert "tensorflow run_clip"

This reverts commit 261f86ac1f.

* fix: duplicated argument
 2023-08-07 16:32:25 +02:00
Yih-Dar
149cb0cce2
Add token arugment in example scripts (#25172) 
* fix

* fix

* fix

* fix

* fix

* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-08-02 11:17:31 +02:00
Yih-Dar
d53b8ad780
Update use_auth_token -> token in example scripts (#25167) 
* pytorch examples

* tensorflow examples

* flax examples

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-07-28 15:33:45 +02:00
Lucain
6232c380f2
Fix .push_to_hub and cleanup get_full_repo_name usage (#25120) 
* Fix .push_to_hub and cleanup get_full_repo_name usage

* Do not rely on Python bool conversion magic

* request changes
 2023-07-28 11:40:08 +02:00
Zach Mueller
aa1b09c5d1
Change logic for logging in the examples (#24956) 
Change logic
 2023-07-20 12:30:10 -04:00
Sylvain Gugger
e9ad51306f
4.32.0.dev0 2023-07-17 13:30:44 -04:00
Ethan
f7d80cb3d2
Fix steps bugs in no trainer examples (#24197) 
Fix step bugs in no trainer + load checkpoint + grad acc
 2023-06-12 11:49:55 -04:00
Sylvain Gugger
ba695c1efd
v4.31.0.dev0 2023-06-07 16:49:00 -04:00
Zachary Mueller
072188d638
Act on deprecations in Accelerate no_trainer examples (#24053) 
Act on deprecation
 2023-06-06 13:04:38 -04:00
Zachary Mueller
b191d7db44
Update all no_trainer with skip_first_batches (#23664) 2023-05-22 14:49:31 -04:00
Sylvain Gugger
a0c0a78233
v4.30.0.dev0 2023-05-09 14:59:38 -04:00
Sebastian
1a8f61110e
fix: Update run_qa.py to work with deepset/germanquad (#23225) 
Call str on id to make sure any ints are converted into the expected format for squad datasets
 2023-05-09 09:20:10 -04:00
Sylvain Gugger
888c4a2ae0
v4.29.0.dev0 2023-04-12 20:04:29 -04:00
Sylvain Gugger
1b1867d86b
Replace -100s in predictions by the pad token (#22693) 
* Replace -100s in predictions by the pad token

* Style

* Try to catch them all
 2023-04-11 09:32:20 -04:00
Sylvain Gugger
ebdb185bef
v4.28.0.dev0 2023-03-14 13:49:10 -04:00
Sylvain Gugger
b19d64d852
Respect documentation on passive log level (#21700) 
* Respect documentation on passive log level

* Fix test and set log level in examples

* Add doc
 2023-02-22 09:39:18 +01:00
Sylvain Gugger
6f79d26442
Update quality tooling for formatting (#21480) 
* Result of black 23.1

* Update target to Python 3.7

* Switch flake8 to ruff

* Configure isort

* Configure isort

* Apply isort with line limit

* Put the right black version

* adapt black in check copies

* Fix copies
 2023-02-06 18:10:56 -05:00
Sylvain Gugger
7119bb052a
v4.27.0.dev0 2023-01-23 16:52:35 -05:00
Sylvain Gugger
05e72aa0c4
Adapt repository creation to latest hf_hub (#21158) 
* Adapt repository creation to latest hf_hub

* Update all examples

* Fix other tests, add Flax examples

* Address review comments
 2023-01-18 11:14:00 -05:00
Observer46
ff8dcb5efa
Fix arguments passed to predict function in QA Seq2seq training script (#21026) 
fix args passed to predict function
 2023-01-06 07:19:42 -05:00
Wang, Yi
ae06bce888
exclude jit time from the speed metric calculation of evaluation and prediction (#20553) 
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>

Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
 2022-12-06 07:37:01 -05:00
Sylvain Gugger
60d1f31bb0
v4.26.0.dev0 2022-12-01 16:19:33 -05:00
Wang, Yi
d752337baa
QnA example: add speed metric (#20522) 2022-12-01 12:04:19 -05:00
Zachary Mueller
822ae69c1b
Update reqs to include min gather_for_metrics Accelerate version (#20242) 
* Update reqs to include min gather_for_metrics Accelerate version

* Other reqs
 2022-11-15 13:28:00 -05:00
Muhammad Sakib Khan Inan
777b1bfe62
New logging support to "Trainer" Class (ClearML Logger) (#20184) 
* Init Update

* ClearML Callbacks integration

* update corrections

* args reporting updated

* {'tensorboard': False, 'pytorch': False}

* ClearML Tests added

* add clearml

* output_uri=True in Task.init

* reformatted integrations.py

* reformatted and fixed

* IF-ELSE statement issue on "has_clearml" resolved

* Add clearml in main callback docs

* Add additional clearml documentation

* Update src/transformers/integrations.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Accept suggestion

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Accept suggestion

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Small change in comments

* Make style clearml

* Accept suggestion

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Victor Sonck <victor.sonck@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2022-11-15 10:08:59 -05:00
Sylvain Gugger
06886d5a68
Only resize embeddings when necessary (#20043) 
* Only resize embeddings when necessary

* Add comment
 2022-11-03 12:05:04 -04:00
Sylvain Gugger
c3a93d8d82
v4.25.0.dev0 2022-10-31 21:48:40 -04:00
Yifan Yang
94d7c3ba44
[Examples] make default preprocessing_num_workers=1 (#19684) 
* [Examples] make default preprocessing_num_workers=1

* [Examples] revert changes in research projects
 2022-10-17 14:17:01 -04:00
FilipposVentirozos
4ed0fa3676
Fix pytorch seq2seq qa (#19258) 
* fixed typo for SQuAD

* Fixed the preprocess_validation_function function for the labels to reflect the remaining truncated instances

* Rolled back the trainer_seq2seq_qa.py for UnboundLocalError: local variable 'metrics' referenced before assignment

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2022-10-12 08:33:44 -04:00
regisss
bb2cfd1824
Add multi-node conditions in trainer_qa.py and trainer_seq2seq.py (#19502) 
* Add multi-node conditions in trainer_qa.py and trainer_seq2seq.py

* Code improvement
 2022-10-11 22:48:56 -04:00
Lysandre
10100979ed Dev version 2022-10-10 17:25:40 -04:00
Sylvain Gugger
0fc68a7e14
Fix seq2seq QA example 2022-09-28 15:45:49 -04:00
Tatsuki Okada
4a0b958d61
Fix trainer seq2seq qa.py evaluate log and ft script (#19208) 
* fix args option

* fix trainer eval log

* fix out of memory qa script

* do isort, black, flake

* fix tokenize target

* take it back.

* fix: comment
 2022-09-28 10:55:46 -04:00
Lysandre
16913b3c92 Dev version 2022-09-14 14:58:20 -04:00
Rahul A R
00fc9217d1
Fixed bug which caused overwrite_cache to always be True (#19000) 
* fixed bug which caused overwrite_cache to always be True (#18967).

* reformatting changes
 2022-09-13 11:29:48 -04:00
Rahul A R
e9442440fc
streamlining 'checkpointing_steps' parsing (#18755) 2022-08-25 11:00:38 -04:00
Atharva Ingle
d90a36d192
remove check for main process for trackers initialization (#18706) 2022-08-22 11:16:27 -04:00
Zachary Mueller
358fc18613
Add evaluate to examples requirements (#18666) 2022-08-18 10:57:39 -04:00
Rasmus Arpe Fogh Jensen
a765b68aa6
Update no_trainer.py scripts to include accelerate gradient accumulation wrapper (#18473) 
* Added accelerate gradient accumulation wrapper to run_image_classification_no_trainer.py example script

* make fixup changes

* PR comments

* changed input to Acceletor based on PR comment, ran make fixup

* Added comment explaining the sync_gradients statement

* Fixed lr scheduler max steps

* Changed run_clm_no_trainer.py script to use accelerate gradient accum wrapper

* Fixed all scripts except wav2vec2 pretraining to use accelerate gradient accum wrapper

* Added accelerate gradient accum wrapper for wav2vec2_pretraining_no_trainer.py script

* make fixup and lr_scheduler step inserted back into run_qa_beam_search_no_trainer.py

* removed changes to run_wav2vec2_pretraining_no_trainer.py script and fixed using wrong constant in qa_beam_search_no_trainer.py script
 2022-08-08 15:52:47 -04:00
Julien Chaumond
9129fd0377
transformers-cli login => huggingface-cli login (#18490) 
* zero chance anyone's using that constant no?

* `transformers-cli login` => `huggingface-cli login`

* `transformers-cli repo create` => `huggingface-cli repo create`

* `make style`
 2022-08-06 09:42:55 +02:00
Kian Sierra McGettigan
0bf1e1aca4
Update no trainer examples for QA and Semantic Segmentation (#18474) 
* swag_no_trainer updated for with gather_metrics

* Removed unused variable samples_seen

* updated examples with gather_for_metrics
 2022-08-04 13:22:19 -04:00
atturaioe
1f84399171
Migrate metric to Evaluate in Pytorch examples (#18369) 
* Migrate metric to Evaluate in pytorch examples

* Remove unused imports
 2022-08-01 07:40:25 -04:00
Sylvain Gugger
986526a0e4
Replace as_target context managers by direct calls (#18325) 
* Preliminary work on tokenizers

* Quality + fix tests

* Treat processors

* Fix pad

* Remove all uses of  in tests, docs and examples

* Replace all as_target_tokenizer

* Fix tests

* Fix quality

* Update examples/flax/image-captioning/run_image_captioning_flax.py

Co-authored-by: amyeroberts <amy@huggingface.co>

* Style

Co-authored-by: amyeroberts <amy@huggingface.co>
 2022-07-29 08:09:09 -04:00
Lysandre
c89a592e87 Dev version 2022-07-27 17:13:57 +02:00
Zachary Mueller
7c4c6f6084
Fix all is_torch_tpu_available issues (#17936) 
* Fix all is_torch_tpu_available
 2022-06-29 11:03:33 -04:00
Zachary Mueller
75259b44bf
Properly calculate the total train iterations and recalculate num epochs in no_trainer scripts (#17856) 2022-06-23 15:46:01 -04:00
Eran Hirsch
1357038164
Add logits_processor parameter, used by generate, to Seq2SeqTrainer methods evaluate and predict (#17805) 
* Add logits_processor parameter, used by `generate`, to `Seq2SeqTrainer` methods `evaluate` and `predict`

* Add all generate parameters to `Seq2SeqTrainer`, and also to `QuestionAnsweringSeq2SeqTrainer` which overrides it

* Remove `self._num_beams` from trainer classes

* - Run fixup
- Fix "Constraint" not exposed
- Fix synced_gpus to actually read from param

* Use kwargs

* Copy kwargs before making changes to it

* Fix style issues unused imports
 2022-06-22 08:11:39 -04:00
Sylvain Gugger
7c6ec195ad v4.21.0.dev0 2022-06-16 12:20:53 -04:00