riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-31 02:02:21 +06:00
Code Issues Actions 23 Packages Projects Releases Wiki Activity
13,130 Commits 887 Branches 232 Tags 2.1 GiB
e26c6f03be
Commit Graph

13130 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Yih-Dar
e26c6f03be
Fix Wav2Vec2 CI OOM (#24190) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-06-12 11:39:04 +02:00
Yih-Dar
8f093fb799
Avoid OOM in doctest CI (#24139) 
* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-06-10 09:47:38 +02:00
Stas Bekman
0d217f428f
[tests] fix bitsandbytes import issue (#24151) 
fix bitsandbytes import issue
 2023-06-09 21:53:11 -07:00
Lysandre Debut
deff5979fe
Tool types (#24032) 
* Tool types

* Tests + fixes

* Isolate types

* Oops

* Review comments + docs

* Tests + docs

* soundfile -> vision
 2023-06-09 13:34:07 -04:00
Freddie Vargus
061580c82c
Fix typo in streamers.py (#24144) 2023-06-09 17:27:46 +01:00
LiamSwayne
12bb853ccd
[documentation] grammatical fixes in image_classification.mdx (#24141) 
Update image_classification.mdx
 2023-06-09 16:59:44 +01:00
Yih-Dar
d0d1632958
Fix Pipeline CI OOM issue (#24124) 
* fix

* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-06-09 16:49:02 +02:00
Arthur
a7501f6fc6
[BlenderBotSmall] Update doc example (#24092) 
* small tokenizer uses `__start__` and `__end__`

* fix PR doctest
 2023-06-09 16:31:57 +02:00
Arthur
5af3a1aa48
[lamaTokenizerFast] Update documentation (#24132) 
* Update documentation

* nits
 2023-06-09 16:30:20 +02:00
Younes Belkada
62fe753325
[SAM] Fix sam slow test (#24140) 
* fix sam test

* update pipeline typehint
 2023-06-09 16:22:09 +02:00
Yih-Dar
847b47c0ee
Fix XGLM OOM on CI (#24123) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-06-09 15:20:59 +02:00
Yih-Dar
b8fe259f16
Fix SAM OOM issue on CI (#24125) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-06-09 15:07:08 +02:00
Yih-Dar
707023d155
Fix TF Rag OOM issue (#24122) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-06-09 15:03:11 +02:00
Sourab Mangrulkar
f2b918356c
fix bugs with trainer (#24134) 
* fix the deepspeed test failures

* apex fix

* FSDP save ckpt fix

* Update src/transformers/trainer.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

---------

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2023-06-09 17:54:53 +05:30
Joao Gante
be10092e63
Generate: PT's top_p enforces min_tokens_to_keep when it is 1 (#24111) 2023-06-09 13:20:05 +01:00
Matt
03585f3734
Correctly build models and import call_context for older TF versions (#24138) 2023-06-09 13:11:01 +01:00
Younes Belkada
a6d05d55f6
[bnb] Fix bnb config json serialization (#24137) 
* fix bnb config json serialization

* forward contrib credits from discussions

---------

Co-authored-by: Andrechang <Andrechang@users.noreply.github.com>
 2023-06-09 13:41:14 +02:00
Elliott Wang
e2972dffdd
PLAM => PaLM (#24129) 2023-06-09 12:32:16 +01:00
Arthur
535542d38d
[Lllama] Update tokenization code to ensure parsing of the special tokens [core] (#24042) 
* preventllama fast from returning token type ids

* remove type hints

* normalised False
 2023-06-09 09:36:19 +02:00
Yih-Dar
2e2088f24b
Avoid GPT-2 daily CI job OOM (in TF tests) (#24106) 
* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-06-08 18:21:09 +02:00
Serge Panev
9322c24476
Fix typo in Llama docstrings (#24020) 
* Fix typo in Llama docstrings

Signed-off-by: Serge Panev <spanev@nvidia.com>

* Update

Signed-off-by: Serge Panev <spanev@nvidia.com>

* make style

Signed-off-by: Serge Panev <spanev@nvidia.com>

---------

Signed-off-by: Serge Panev <spanev@nvidia.com>
 2023-06-08 17:19:07 +01:00
Radamés Ajna
a73883ae9e
add trust_remote_code option to CLI download cmd (#24097) 
* add trust_remote_code option

* require_torch
 2023-06-08 11:13:57 -04:00
Younes Belkada
8b169142f8
[GPT2] Add correct keys on _keys_to_ignore_on_load_unexpected on all child classes of GPT2PreTrainedModel (#24113) 
* add correct keys on `_keys_to_ignore_on_load_unexpected`

* oops
 2023-06-08 10:21:42 -04:00
Marc Sun
71a114d3e0
fix get_keys_to_not_convert function (#24095) 
* fix get_keys_to_not_convert funct

* Fix style
 2023-06-08 10:14:27 -04:00
Sylvain Gugger
8c5f306719
Update the pin on Accelerate (#24110) 2023-06-08 10:11:01 -04:00
Younes Belkada
2200bf7a45
[Trainer] Correct behavior of _load_best_model for PEFT models (#24103) 
* v1

* some refactor

- add ST format as well

* fix

* add `ADAPTER_WEIGHTS_NAME` & `ADAPTER_SAFE_WEIGHTS_NAME`
 2023-06-08 15:38:30 +02:00
Sourab Mangrulkar
0f23605094
reset accelerate env variables after each test (#24107) 2023-06-08 09:19:07 -04:00
Sadra Barikbin
5fa0a1b23b
Fix a tiny typo in WhisperForConditionalGeneration::generate docstring (#24045) 2023-06-08 13:54:56 +01:00
Sylvain Gugger
ba695c1efd
v4.31.0.dev0 2023-06-07 16:49:00 -04:00
Sylvain Gugger
c3572e6bfb
Add AzureOpenAiAgent (#24058) 
* Add AzureOpenAiAgent

* quality

* Update src/transformers/tools/agents.py

Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>

---------

Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
 2023-06-07 16:34:53 -04:00
Zachary Mueller
5eb3d3c702
Up pinned accelerate version (#24089) 
* Min accelerate

* Also min version

* Min accelerate

* Also min version

* To different minor version

* Empty
 2023-06-07 16:21:51 -04:00
Sourab Mangrulkar
d1c039e398
fix accelerator prepare during eval only mode (#24014) 
* fix mixed precision prep during eval only mode

* update to address comments

* update to reflect the changes in accelerate
 2023-06-08 01:03:13 +05:30
Sylvain Gugger
2c887cf8e0
Do not prepare lr scheduler as it as the right number of steps (#24088) 
* Do not prepare lr scheduler as it as the right number of steps

* Trigger CI

* Trigger CI

* Trigger CI

* Add fake comment

* Remove fake comment

* Trigger CI please!
 2023-06-07 15:31:32 -04:00
Sourab Mangrulkar
12298cb65c
fix executable batch size issue (#24067) 
* fix executable batch size issue

* fix

* undo
 2023-06-07 22:08:04 +05:30
Mishig
ef010071ee
Update delete_doc_comment_trigger.yml (#24084) 
fix base workflow name
 2023-06-07 17:55:48 +02:00
Sylvain Gugger
89b00eef94
Fix expected value in tests of the test fetcher (#24077) 
* Fix expected value in tests of the test fetcher

* Fix trigger for repo util tests
 2023-06-07 11:38:56 -04:00
Mishig
5c9394b54c
[doc build] Use secrets (#24079) 2023-06-07 17:33:39 +02:00
Matt
1fc832b454
Make the TF dummies even smaller (#24071) 
* Let's see if we can use the smallest possible dummies

* Make GPT-2's dummies a little longer

* Just use (1,2) as the default shape

* Update other dummies in sync

* Correct imports for Keras 2.13

* Shrink the Wav2Vec2 dummies
 2023-06-07 16:23:05 +01:00
Yih-Dar
092c14c37d
Be nice to TF (#24076) 
* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-06-07 16:18:13 +02:00
Younes Belkada
4795219228
[bnb] Fix bnb skip modules (#24043) 
* fix skip modules test

* oops

* address comments
 2023-06-07 15:27:46 +02:00
Michael Benayoun
a1160185ff
Fix is_optimum_neuron_available (#23961) 
Fix is_optimum_neuron_available
 2023-06-07 09:13:01 -04:00
Younes Belkada
6b548129b1
[Hub] Add safe_serialization in push_to_hub (#24074) 
add `safe_serialization` in push_to_hub
 2023-06-07 09:07:33 -04:00
Younes Belkada
6daf7c311b
Support PEFT models when saving the model using trainer (#24073) 
* support PEFT models when saving the model using trainer

* fixup
 2023-06-07 14:30:55 +02:00
YangLiu
1e4a7737ed
Add support for non-rust implemented tokenization for __getitem__ method. (#24039) 
* Add support for non-rust implemented tokenization for `__getitem__` method.

* Update for error message on adding new sub-branch for `__item__` method.

---------

Co-authored-by: liuyang17 <liuyang17@zhihu.com>
 2023-06-07 12:29:19 +01:00
Patrick von Platen
52972e70c7
[Wav2Vec2] Fix torch srcipt (#24062) 
* [Wav2Vec2] Fix torch srcipt

* fix more
 2023-06-07 07:27:07 -04:00
Joao Gante
612b2a1a6d
Generate: increase left-padding test atol (#23448) 
increase atol
 2023-06-07 11:56:57 +01:00
Sylvain Gugger
f1660d7e23
Remote code improvements (#23959) 
* Fix model load when it has both code on the Hub and locally

* Add input check with timeout

* Add tests

* Apply suggestions from code review

Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>

* Some non-saved stuff

* Add feature extractors

* Add image processor

* Add model

* Add processor and tokenizer

* Reduce timeout

---------

Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
 2023-06-06 14:31:14 -04:00
Sylvain Gugger
60825f2c6e
Fix device placement for model-parallelism in generate for encoder/de… (#24025) 
* Fix device placement for model-parallelism in generate for encoder/decoders

* Remove debug statements
 2023-06-06 14:30:59 -04:00
Yih-Dar
02d255db26
bring back filtered_test_list_cross_tests.txt (#24055) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-06-06 19:35:24 +02:00
Edward Z. Yang
bc9ecef942
Use new parametrization based weight norm if available (#24030) 
* Use new parametrization based weight norm if available

See https://github.com/pytorch/pytorch/pull/103001

Signed-off-by: Edward Z. Yang <ezyang@meta.com>

* handle copies

Signed-off-by: Edward Z. Yang <ezyang@meta.com>

* black

Signed-off-by: Edward Z. Yang <ezyang@meta.com>

---------

Signed-off-by: Edward Z. Yang <ezyang@meta.com>
 2023-06-06 13:34:57 -04:00