riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-31 02:02:21 +06:00
Code Issues Actions 23 Packages Projects Releases Wiki Activity
12,916 Commits 887 Branches 232 Tags 2.1 GiB
a7920065f2
Commit Graph

12916 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Boda Sadallah
a7920065f2
fix bug in group_texts function, that was inserting short batches (#23429) 
* fix bug in group_texts function, that was inserting short batches

* fully exclude short batches and return empty dict instead

* fix style
 2023-05-18 14:22:30 -04:00
Sylvain Gugger
b7b81d9344
Clean up CUDA kernels (#23455) 2023-05-18 14:14:43 -04:00
Sylvain Gugger
40ed18ae15
Add an option to log result from the Agent (#23454) 2023-05-18 14:06:49 -04:00
Jonas Mueller
f69589d1bc
add cleanlab to awesome-transformers tools list (#23440) 
* add tool to awesome-transformers list

* add keyword list

* sgugger wording suggestion

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

---------

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2023-05-18 13:14:28 -04:00
Sylvain Gugger
167aa76cfa
Properly guard PyTorch stuff (#23452) 
* Properly guard PyTorch stuff

* [all-test]

* [all-test] Fix model imports as well

* Making sure StoppingCriteria is always defined

* [all-test]
 2023-05-18 12:17:17 -04:00
Yih-Dar
ffad4f1373
Update tiny models and pipeline tests (#23446) 
* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-05-18 17:29:04 +02:00
Yih-Dar
2406dbdcfa
Less flaky test_assisted_decoding_matches_greedy_search (#23451) 
* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-05-18 17:28:22 +02:00
Yih-Dar
21f7e81b6b
Make RwkvModel accept attention_mask but discard it internally (#23442) 
* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-05-18 17:14:25 +02:00
Sylvain Gugger
cf43200861
Add local agent (#23438) 
* Add local agent

* Document LocalAgent
 2023-05-18 11:09:55 -04:00
Joao Gante
db13634183
TF: GPT2 with native embedding layers (#23436) 2023-05-18 14:46:40 +01:00
joaoareis
c618ab4fab
Fix DecisionTransformerConfig doctring (#23450) 2023-05-18 14:07:10 +01:00
Yih-Dar
5777c3cb3f
Fix (skip) a pipeline test for RwkvModel (#23444) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-05-18 14:54:23 +02:00
Nayeon Han
8cfae44093
🌐 [i18n-KO] Translated tasks/zero_shot_object_detection.mdx to Korean (#23430) 
docs: ko: zero_shot_object_detection
 2023-05-18 08:52:17 -04:00
Chris Hammill
f2d2880bbb
remove unnecessary print in gpt neox sequence classifier (#23433) 2023-05-18 11:34:33 +01:00
Joao Gante
aea7b23b57
Generate: skip left-padding tests on old models (#23437) 2023-05-18 11:04:51 +01:00
Yih-Dar
a8732e09bb
Fix device issue in SwiftFormerModelIntegrationTest::test_inference_image_classification_head (#23435) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-05-17 19:48:18 +02:00
Hugo Abonizio
0f2c738207
Remove hardcoded prints in Trainer (#23432) 2023-05-17 13:08:12 -04:00
Joao Gante
a574de302f
Encoder-Decoder: add informative exception when the decoder is not compatible (#23426) 2023-05-17 17:42:54 +01:00
Yih-Dar
939a65aba7
Update Bigbird Pegasus tests (#23431) 
* fix

* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-05-17 18:14:29 +02:00
Joao Gante
cf9e7cb079
TF: embeddings out of bounds check factored into function (#23427) 2023-05-17 17:04:51 +01:00
Zachary Mueller
45e3d6496a
Update error message when Accelerate isn't installed (#23373) 
Update error
 2023-05-17 11:16:02 -04:00
Lysandre Debut
ea0eb15649
Small fixes and link in the README (#23428) 
Fix + link
 2023-05-17 11:07:36 -04:00
Lysandre Debut
5ba0c332b6
Top 100 (#22912) 
* Awesome Transformers

* Update

* Update

* Keywords

* Keywords

* Complete document

* Add lm-evaluation-harness

* Edit txtai according to David's comments

* Update awesome-transformers.md
 2023-05-17 10:46:55 -04:00
IMvision12
ebb649a4e3
Add Missing tokenization test [electra] (#22997) 
* Create test_tokenization_electra.py

* Update tests/models/electra/test_tokenization_electra.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

---------

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2023-05-17 10:45:15 -04:00
cyy
a2789adddf
[Reland] search model buffers for dtype as the last resort (#23319) 
search model buffers for dtype as the last resort
 2023-05-17 09:05:07 -04:00
Taras Tsugrii
3d764fe860
Return early once stop token is found. (#23421) 
Previously even after finding a stop token, other stop tokens were considered, which is unnecessary and slows down processing.

Currently, this unnecessary overhead is negligible since there are usually 2 stop tokens considered and they are fairly short, but in future it may become more expensive.
 2023-05-17 09:00:08 -04:00
Younes Belkada
3d3c7d4213
[SAM] fix sam slow test (#23376) 
* fix sam slow test

* oops

* fix error message
 2023-05-17 14:27:43 +02:00
Yih-Dar
22a0769933
Update 3 docker files to use cu118 (#23406) 
* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-05-17 14:26:50 +02:00
Taras Tsugrii
a6c9643ce7
Use dict.items to avoid unnecessary lookups. (#23415) 
It's more efficient to iterate over key, value dict pairs instead of iterating over keys and performing value lookups on each iteration. It's also more idiomatic.
 2023-05-17 11:25:29 +01:00
Taras Tsugrii
43f146208e
Fix a typo in HfAgent docstring. (#23420) 2023-05-17 09:43:02 +01:00
Yih-Dar
46d2468695
Update ConvNextV2ModelIntegrationTest::test_inference_image_classification_head (#23402) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-05-16 23:35:11 +02:00
Yih-Dar
ca3df9f0cf
Run doctest (in PRs) only when some doc example(s) are modified (#23387) 
* fix

* fix

* update

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-05-16 23:29:02 +02:00
ropoctl
17d0290e57
Why crash the whole run when HFHub gives a 50x error? (#23320) 
Logging an error and continuing is probably following the principle of least surprise.
 2023-05-16 15:46:53 -04:00
Sylvain Gugger
d712ebd86d
Fix smdistributed check (#23414) 2023-05-16 15:18:31 -04:00
Taras Tsugrii
4e244b8817
Replace appends with list comprehension. (#23359) 
It's more idiomatic and significantly more efficient because
1) it avoids repeated `append` call that Python has to resolve on each iteration
2) can preallocate the size of the final list avoiding resizing
 2023-05-16 20:14:11 +01:00
Joao Gante
918a06e25d
Generate: add test to check KV format (#23403) 
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2023-05-16 19:28:19 +01:00
Sylvain Gugger
9cf4a8b456
Build with non Python files (#23405) 
* Add a test of the built release

* Polish everything

* Trigger CI
 2023-05-16 14:23:10 -04:00
Joao Gante
5b1ad0eb73
Docs: add link to assisted generation blog post (#23397) 2023-05-16 18:54:34 +01:00
Stas Bekman
bbbc5c15d4
[AutoModel] fix torch_dtype=auto in from_pretrained (#23379) 
* [automodel] fix torch_dtype=auto in from_pretrained

* add test

* fix logic

* Update src/transformers/models/auto/auto_factory.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

---------

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2023-05-16 10:21:42 -07:00
Zachary Mueller
8a58809312
Fix translation no_trainer (#23407) 
* Fix translation
 2023-05-16 13:10:42 -04:00
Joao Gante
130e154291
Generate: faster can_generate check on TF and Flax (#23398) 2023-05-16 15:12:21 +01:00
Younes Belkada
2922e394e3
[Pix2Struct] Add conditional generation on docstring example (#23399) 
add conditional generation on docstring
 2023-05-16 15:59:18 +02:00
Lucain
52d516c3a9
Minor fixes in transformers-tools (#23364) 
* Few fixes in new Tools implementation

* code quality
 2023-05-16 15:55:44 +02:00
Sohyun Sim
728c5e82cc
🌐 [i18n-KO] Translated asr.mdx to Korean (#23106) 
* docs: ko: task/asr.mdx

* feat: manual draft

* fix: resolve suggestions

Co-authored-by: Hyeonseo Yun <0525yhs@gmail.com>

---------

Co-authored-by: Hyeonseo Yun <0525yhs@gmail.com>
 2023-05-16 09:22:56 -04:00
Ivan Sedykh
770a1275d3
Fix chat prompt in HFAgent (#23335) 
fix chat prompts
 2023-05-16 09:18:58 -04:00
Joao Gante
466af1a356
OPT/BioGPT: Improved attention mask shape exception (#23270) 2023-05-16 13:59:53 +01:00
Yih-Dar
21741e8c7e
Update test_batched_inference_image_captioning_conditioned (#23391) 
* fix

* fix

* fix test + add more docs

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: younesbelkada <younesbelkada@gmail.com>
 2023-05-16 14:49:24 +02:00
Yih-Dar
d765717c76
Fix RwkvModel (#23392) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-05-16 12:14:54 +02:00
ready-research
80ca924709
Use mkstemp to replace deprecated mktemp (#23372) 
* Use `mkstemp` to replace deprecated `mktemp`

The `tempfile.mktemp` function is [deprecated](https://docs.python.org/3/library/tempfile.html#tempfile.mktemp) due to [security issues](https://cwe.mitre.org/data/definitions/377.html).

* Update src/transformers/utils/hub.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

---------

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2023-05-16 11:10:54 +01:00
Koki Tanaka
ba6815e824
Replace NumPy Operations with JAX NumPy Equivalents for JIT Compilation Compatibility (#23356) 
* Replace numpy operations with jax.numpy for JIT compatibility

Replaced numpy operations with their jax.numpy equivalents in the transformer library. This change was necessary to prevent errors during JIT compilation. Specifically, the modifications involve changing numpy's in-place assignments to jax.numpy's immutable update methods.

* rm numpy import

* rm numpy import and fix np->jnp

* fixed slices bug

* fixed decoder_start_tokens -> decoder_start_token_id

* fixed jnp in modleing mt5

* doc fix

* rm numpy import

* make
 2023-05-16 10:54:19 +01:00