riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-05 13:50:13 +06:00
Code Issues Actions 26 Packages Projects Releases Wiki Activity
11,717 Commits 861 Branches 225 Tags 2.1 GiB
ff8dcb5efa
Commit Graph

318 Commits

Author SHA1 Message Date
Observer46
ff8dcb5efa
Fix arguments passed to predict function in QA Seq2seq training script (#21026) 
fix args passed to predict function
 2023-01-06 07:19:42 -05:00
Roy Hvaara
35a7052b61
[NumPy] Remove references to deprecated NumPy type aliases (#21022) 
[NumPy] Remove references to deprecated NumPy type aliases.

This change replaces references to a number of deprecated NumPy type aliases (np.bool, np.int, np.float, np.complex, np.object, np.str) with their recommended replacement (bool, int, float, complex, object, str).

NumPy 1.24 drops the deprecated aliases, so we must remove uses before updating NumPy.

Co-authored-by: Peter Hawkins <phawkins@google.com>

Co-authored-by: Peter Hawkins <phawkins@google.com>
 2023-01-05 13:02:10 -05:00
Magnus Pierrau
1d21471c78
Added mask_time_prob and mask_time_length arguments to wav2vec2 pretraining script (#20985) 
Added mask_time_prob and mask_time_length arguments to wav2vec2 pretraining script and readme - new branch
 2023-01-05 16:24:55 +00:00
Wang, Yi
9c9fe89f84
[run_clm example] add torch_dtype option for model load. (#20971) 
* [run_clm example] add torch_dtype option for model load.
for BLOOM 175B model. peak memory will reduce about 350G for inference. the weight of BLOOM in model hub is bfloat16

Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>

* add other type in option

* fix style

Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
 2023-01-03 09:33:11 -05:00
Márton Makrai
3830b3f74a
Fixes typo in the help text for --max_length (#20883) 2022-12-24 02:07:06 -05:00
NielsRogge
d87e381f93
[Examples] Update big table (#20845) 
Update big table

Co-authored-by: Niels Rogge <nielsrogge@Nielss-MBP.localdomain>
 2022-12-21 11:34:31 +01:00
Emmanuel Schmidbauer
0526a075c5
run_speech_recognition_seq2seq.py: add cache_dir param to dataset (#20540) 2022-12-07 18:23:16 +00:00
Francisco Kurucz
f821bea0ad
Fix link to speech encoder decoder model in speech recognition readme (#20633) 2022-12-06 15:46:41 -05:00
Wang, Yi
ae06bce888
exclude jit time from the speed metric calculation of evaluation and prediction (#20553) 
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>

Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
 2022-12-06 07:37:01 -05:00
Sylvain Gugger
60d1f31bb0
v4.26.0.dev0 2022-12-01 16:19:33 -05:00
Wang, Yi
d752337baa
QnA example: add speed metric (#20522) 2022-12-01 12:04:19 -05:00
Zachary Mueller
9d1ef009b8
Fix flakey test with seed (#20318) 2022-11-18 11:33:25 -05:00
Sanchit Gandhi
c29a2f7c9c
[ASR Examples] Update README for Whisper (#20230) 
* [ASR Examples] Update README for seq2seq

* add language info

* add training results

* re-word
 2022-11-18 11:24:25 +00:00
Zachary Mueller
441811ecd7
Fix summarization script (#20286) 2022-11-16 15:57:07 -05:00
Jiahao Li
9681f052a1
Fix result saving errors of pytorch examples (#20276) 2022-11-16 09:51:04 -05:00
Zachary Mueller
822ae69c1b
Update reqs to include min gather_for_metrics Accelerate version (#20242) 
* Update reqs to include min gather_for_metrics Accelerate version

* Other reqs
 2022-11-15 13:28:00 -05:00
Muhammad Sakib Khan Inan
777b1bfe62
New logging support to "Trainer" Class (ClearML Logger) (#20184) 
* Init Update

* ClearML Callbacks integration

* update corrections

* args reporting updated

* {'tensorboard': False, 'pytorch': False}

* ClearML Tests added

* add clearml

* output_uri=True in Task.init

* reformatted integrations.py

* reformatted and fixed

* IF-ELSE statement issue on "has_clearml" resolved

* Add clearml in main callback docs

* Add additional clearml documentation

* Update src/transformers/integrations.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Accept suggestion

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Accept suggestion

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Small change in comments

* Make style clearml

* Accept suggestion

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Victor Sonck <victor.sonck@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2022-11-15 10:08:59 -05:00
Yih-Dar
cf7b98b807
Fix run_clip.py (#20234) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-11-15 15:45:21 +01:00
Ming Liu
36b063ed4f
Update README.md (#20188) 
There is typo in the original hyperlink.

Below is the original version:
Based on the script [`run_translation_no_trainer.py`](https://github.com/huggingface/transformers/blob/main/examples/pytorch/translation/**run_translationn_no_trainer.py**).
 2022-11-14 12:53:02 -05:00
Sanchit Gandhi
af1a7c8ca3
[Examples] Generalise Seq2Seq ASR to handle Whisper (#19519) 
* merge conflicts

* bos and eos in datacollator

* (temp) hardcode removal of attention mask

* freeze encoder

* actually freeze encoder

* set max length / num beams according to gen kwargs

* (temp) fix tests

* don't pop attn mask

* override return attention mask config from Hub

* Hub configs updated 🤗

* final fixes

* update type annotations

* backward comp
 2022-11-14 17:45:46 +00:00
bhuang
3502c202f9
Update README.md (#20063) 2022-11-04 08:56:54 -04:00
Sylvain Gugger
06886d5a68
Only resize embeddings when necessary (#20043) 
* Only resize embeddings when necessary

* Add comment
 2022-11-03 12:05:04 -04:00
amyeroberts
a6b7759880
Add Image Processors (#19796) 
* Add CLIP image processor

* Crop size as dict too

* Update warning

* Actually use logger this time

* Normalize doesn't change dtype of input

* Add perceiver image processor

* Tidy up

* Add DPT image processor

* Add Vilt image processor

* Tidy up

* Add poolformer image processor

* Tidy up

* Add LayoutLM v2 and v3 imsge processors

* Tidy up

* Add Flava image processor

* Tidy up

* Add deit image processor

* Tidy up

* Add ConvNext image processor

* Tidy up

* Add levit image processor

* Add segformer image processor

* Add in post processing

* Fix up

* Add ImageGPT image processor

* Fixup

* Add mobilevit image processor

* Tidy up

* Add postprocessing

* Fixup

* Add VideoMAE image processor

* Tidy up

* Add ImageGPT image processor

* Fixup

* Add ViT image processor

* Tidy up

* Add beit image processor

* Add mobilevit image processor

* Tidy up

* Add postprocessing

* Fixup

* Fix up

* Fix flava and remove tree module

* Fix image classification pipeline failing tests

* Update feature extractor in trainer scripts

* Update pad_if_smaller to accept tuple and int size

* Update for image segmentation pipeline

* Update src/transformers/models/perceiver/image_processing_perceiver.py

Co-authored-by: Alara Dirik <8944735+alaradirik@users.noreply.github.com>

* Update src/transformers/image_processing_utils.py

Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>

* Update src/transformers/models/beit/image_processing_beit.py

Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>

* PR comments - docstrings; remove accidentally added resize; var names

* Update docstrings

* Add exception if size is not in the right format

* Fix exception check

* Fix up

* Use shortest_edge in tuple in script

Co-authored-by: Alara Dirik <8944735+alaradirik@users.noreply.github.com>
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
 2022-11-02 11:57:36 +00:00
Sylvain Gugger
c3a93d8d82
v4.25.0.dev0 2022-10-31 21:48:40 -04:00
Sanchit Gandhi
f38a145418
[ASR] Update 'tasks' for model card (#19986) 2022-10-31 16:50:17 +00:00
regisss
5d2d51a0fb
Fix LR (#19875) 2022-10-26 08:35:53 -04:00
GMFTBY
71786b10c5
Adding the state-of-the-art contrastive search decoding methods for the codebase of generation_utils.py (#19477) 
* add: the contrastive search for generaton_utils

* add: testing scripts for contrastive search under examples/text-generation

* update the quality of codes

* revise the docstring; make the generation_contrastive_search.py scripts;

* revise the examples/pytorch/text-generation/run_generation_contrastive_search.py to the auto-APIs format

* revise the necessary documents

* fix: revise the docstring of generation_contrastive_search.py

* Fix the code indentation

* fix: revise the nits and examples in contrastive_search docstring.

* fix the copyright

* delete generation_contrastive_search.py

* revise the logic in contrastive_search

* update the intergration test and the docstring

* run the tests over

* add the slow decorate to the contrastive_search intergrate test

* add more test

* do the style, quality, consistency checks
 2022-10-19 10:17:46 +01:00
amyeroberts
31ec424b3d
Add decorator to flaky test (#19674) 2022-10-18 18:51:37 +01:00
Yifan Yang
94d7c3ba44
[Examples] make default preprocessing_num_workers=1 (#19684) 
* [Examples] make default preprocessing_num_workers=1

* [Examples] revert changes in research projects
 2022-10-17 14:17:01 -04:00
Sanchit Gandhi
eefcecaa35
[Examples] Fix typos in run speech recognition seq2seq (#19514) 2022-10-12 15:33:22 +01:00
FilipposVentirozos
4ed0fa3676
Fix pytorch seq2seq qa (#19258) 
* fixed typo for SQuAD

* Fixed the preprocess_validation_function function for the labels to reflect the remaining truncated instances

* Rolled back the trainer_seq2seq_qa.py for UnboundLocalError: local variable 'metrics' referenced before assignment

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2022-10-12 08:33:44 -04:00
regisss
bb2cfd1824
Add multi-node conditions in trainer_qa.py and trainer_seq2seq.py (#19502) 
* Add multi-node conditions in trainer_qa.py and trainer_seq2seq.py

* Code improvement
 2022-10-11 22:48:56 -04:00
Lysandre
10100979ed Dev version 2022-10-10 17:25:40 -04:00
wei zhao
7d5ce6802e
Fix typo in image-classification/README.md (#19424) 
Fix link typo of the following content.
PyTorch version, Trainer
PyTorch version, no Trainer
 2022-10-10 09:16:58 -04:00
ddobokki
fa4bcd5274
edit: cast attention_mask to long in DataCollatorCTCWithPadding (#19369) 
* edit: casting attention_mask to long in DataCollatorCTCWithPadding

* edit: casting attention_mask to long in DataCollatorCTCWithPadding
 2022-10-07 10:05:48 -04:00
Zachary Mueller
ad98642a82
Fix gather for metrics (#19360) 2022-10-05 14:52:01 -04:00
Divyanshu Kumar
c28d04e9e2
Update no_trainer script for summarization (#19277) 
* Update no_trainer script for summarization

* removed unnecessary import

* fixes notation mistake

* removed: unused variable
 2022-10-03 09:21:51 -04:00
Sylvain Gugger
0fc68a7e14
Fix seq2seq QA example 2022-09-28 15:45:49 -04:00
Tatsuki Okada
4a0b958d61
Fix trainer seq2seq qa.py evaluate log and ft script (#19208) 
* fix args option

* fix trainer eval log

* fix out of memory qa script

* do isort, black, flake

* fix tokenize target

* take it back.

* fix: comment
 2022-09-28 10:55:46 -04:00
Sylvain Gugger
c20b2c7e18
Use repo_type instead of deprecated datasets repo IDs (#19202) 
* Use repo_type instead of deprecated datasets repo IDs

* Add missing one in doc
 2022-09-26 09:50:48 -04:00
Enze
5da6afdd8d
Update run_clip.py (#19130) 
The overwrite_cache parameter is declared twice.
 2022-09-23 20:48:41 +02:00
Leandro von Werra
ef6741fe65
Fix GLUE MNLI when using max_eval_samples (#18722) 2022-09-21 09:33:22 +02:00
Santiago Castro
06f341de4f
Add a missing space in a script arg documentation (#19113) 2022-09-20 21:43:32 +02:00
Lysandre
16913b3c92 Dev version 2022-09-14 14:58:20 -04:00
Rahul A R
00fc9217d1
Fixed bug which caused overwrite_cache to always be True (#19000) 
* fixed bug which caused overwrite_cache to always be True (#18967).

* reformatting changes
 2022-09-13 11:29:48 -04:00
Rafał Jankowski
85125fcffd
Neptune.ai integration improvements (#18934) 
* NeptuneCallback improvements

* After review suggestions and deduplication of initial run

* Added volatile checkpoints support due to missing post-rebase commit

* Update README per review comments

- Remove list formatting
- Correct Neptune docs link

Co-authored-by: Sabine <sabine.nyholm@neptune.ai>
 2022-09-09 11:37:34 -04:00
Nicholas Broad
4f299b2446
Accelerator end training (#18910) 
* add accelerator.end_training()

Some trackers need this to end their runs.

* fixup and quality

* add space

* add space again ?!?
 2022-09-07 07:46:26 -04:00
arun99481
3b19c0317b
updating gather function with gather_for_metrics in run_wav2vec2_pretraining (#18877) 
Co-authored-by: Arun Rajaram <arunrajaram@Aruns-MacBook-Pro.local>
 2022-09-06 07:36:37 -04:00
Sylvain Gugger
c61f116b63
Tie weights after preparing the model in run_clm (#18855) 2022-09-01 12:06:56 -04:00
Rahul A R
e9442440fc
streamlining 'checkpointing_steps' parsing (#18755) 2022-08-25 11:00:38 -04:00