riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-31 02:02:21 +06:00
Code Issues Actions 23 Packages Projects Releases Wiki Activity
11,858 Commits 887 Branches 232 Tags 2.1 GiB
94a7edd938
Commit Graph

11858 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Arthur
94a7edd938
[GenerationConfig] add additional kwargs handling (#21269) 
* add additional kwargs handling

* fix issue when serializing

* correct order of kwargs removal for serialization in from dict

* add `dict_torch_dtype_to_str` in case a dtype is needed for generation

* add condition when adding the kwargs : not from config

* Add comment based on review

Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>

* add test function

* default None when poping arg

Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
 2023-01-24 19:04:42 +01:00
Stas Bekman
9286039c2a
[examples/deepspeed] fix renamed api (#21283) 2023-01-24 09:54:33 -08:00
Younes Belkada
e2e393c6f2
[t5] Fix T5 inference in float16 + bnb error (#21281) 
* attempts to fix:

- upcast input for `T5DenseActDense`
- add the condition `self.wo.weight.dtype != torch.int8`
- added tests on `test/mixed_int8`
- `make fixup`

* fix ci test
 2023-01-24 18:14:38 +01:00
Alara Dirik
f424b09410
Fix MaskFormerImageProcessor.post_process_instance_segmentation (#21256) 
* fix instance segmentation post processing

* add Mask2FormerImageProcessor
 2023-01-24 18:49:29 +03:00
Hirokazu Kiyomaru
767939af52
Use logger.info instead of print to emit a logging message in hub.py (#21273) 
use logger.info() instead of print() to emit a debug message
 2023-01-24 10:37:10 -05:00
Nicolas Patry
67316444b0
Hotifx remove tuple for git config image processor. (#21278) 2023-01-24 16:07:50 +01:00
Matt
071529bd54
Use return_tensors="np" instead of "tf" (#21266) 
Return NP instead of TF tensors for our data loading pipeline
 2023-01-24 13:37:49 +00:00
Younes Belkada
f0fc791298
[Doc] fix broken link (#21276) 
fix broken link
 2023-01-24 11:18:48 +01:00
Yih-Dar
bde7378bf0
Skip test_multi_gpu_data_parallel_forward for UperNetModelTest (#21216) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-01-24 10:41:16 +01:00
Sylvain Gugger
7119bb052a
v4.27.0.dev0 2023-01-23 16:52:35 -05:00
Sylvain Gugger
fd5cdaeea6
Models docstring (#21225) 
* Clean all models

* Style

* Last to remove

* address review comments

* Address review comments
 2023-01-23 14:33:18 -05:00
Maria Khalusova
9e86c4e193
Supported pipeline tasks update (#21268) 
* added tasks from SUPPORTED_TASKS to docstrings

* make style

* sorted the tasks in the docstrtings in alphabetical order
 2023-01-23 14:23:20 -05:00
Arthur
d8415ba42e
[Whisper] fix all issues with unk token (#21250) 
* fix all issues with unk token

* fixup
 2023-01-23 20:19:57 +01:00
amyeroberts
c18b4fbe9f
Add class properties with warnings (#21195) 
* Replace reduce_labels with do_reduce_labels

* Replace only for __init__ and preprocess

* Add class properties with warnings

* Update tests
 2023-01-23 18:45:27 +00:00
Arthur
b80b2218b5
[ci-daily] Fix pipeline tests (#21257) 
* use streaming dataset

* fix whisper's test

* add rescale argument to chunk_iter
 2023-01-23 19:32:49 +01:00
Maria Khalusova
275ad9d80a
Add: TensorFlow example for semantic segmentation task guide (#21223) 
* wip: adding tf example for semantic segmentation guide

* completed the working example in tf

* make style

* Update docs/source/en/tasks/semantic_segmentation.mdx

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/tasks/semantic_segmentation.mdx

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* fixed a callback doc links

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2023-01-23 13:32:15 -05:00
Maria Khalusova
2218dac5d2
Notebook examples grouping and update (#21265) 
* Split the examples by modality, added missing examples

* fixed a link
 2023-01-23 12:51:24 -05:00
amyeroberts
e2bd7f80d0
Update tests: replace feature extractor tests with image processor (#20768) 
* Update imports and test fetcher

* Revert but keep test fetcher update

* Fix imports

* Fix all imports

* Replace fe with ip names

* Add generate kwargs to `AutomaticSpeechRecognitionPipeline` (#20952)

* Add generate kwargs to AutomaticSpeechRecognitionPipeline

* Add test for generation kwargs

* Update image processor parameters if creating with kwargs (#20866)

* Update parameters if creating with kwargs

* Shallow copy to prevent mutating input

* Pass all args in constructor dict - warnings in init

* Fix typo

* Rename tester class

* Rebase and tidy up

* Fixup

* Use ImageProcessingSavingTestMixin

* Update property ref in tests

* Update property ref in tests

* Update recently merged in models

* Small fix

Co-authored-by: bofeng huang <bofenghuang7@gmail.com>
 2023-01-23 17:25:41 +00:00
amyeroberts
354ea44340
Replace reduce_labels with do_reduce_labels (#21218) 
* Replace reduce_labels with do_reduce_labels

* Replace only for __init__ and preprocess

* Update tests
 2023-01-23 17:21:33 +00:00
Joao Gante
1eda4a4102
Generate: save generation config with the models' .save_pretrained() (#21264) 2023-01-23 16:21:44 +00:00
amyeroberts
cf1a1eed70
Add missing checkpoint for doctest (#21258) 2023-01-23 15:27:25 +00:00
Mostafa Elhoushi
5603f78fc4
Add scikit-learn dependency to train langage-modeling (#21229) 2023-01-23 09:54:45 -05:00
Kambe Hiroyuki
929111698c
Add Japanese translation installation.mdx (#21241) 
* Add Japanese translation installation.mdx

* Fixed for consistency with english version
 2023-01-23 15:38:30 +01:00
Yih-Dar
cb6b56859a
Fix reformer CI (#21254) 
* fix ReformerForSequenceClassification doc example

* fix ReformerForMaskedLM doc example

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-01-23 15:34:14 +01:00
raghavanone
eaace0c668
Optimize by not computing gradients for parameters set to requires_grad=False (#21236) 
* Optimize by not computing gradients for parameters set to requires_grad=False

* Make change to retrigger the build

* Fix isort issue

* Fix issue
 2023-01-23 09:27:59 -05:00
NielsRogge
6e4d3f0859
[GIT] Convert more checkpoints (#21245) 
* Extend conversion script

* Remove print statement

Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
 2023-01-23 15:19:27 +01:00
amyeroberts
66459ce319
Add test_image_processing_common.py (#20785) 
* Add test_image_processing_common.py

* Fix typo

* Update imports and test fetcher

* Revert but keep test fetcher update

* Fix imports

* Fix all imports

* Formatting fix

* Update tests/test_image_processing_common.py
 2023-01-23 13:48:30 +00:00
Ogundepo Odunayo
96b2b2de12
Extend Script to enable conversion of Encoder Only T5x Models to Pytorch (#20907) 
* add converter for t5x_retrieval model

* update args

* Update src/transformers/models/t5/convert_t5x_checkpoint_to_pytorch.py

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* style  editing -> convert t5x to pytorch

* make style

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
 2023-01-23 14:41:43 +01:00
NielsRogge
91ff7efeeb
[DETR and friends] Use AutoBackbone as alternative to timm (#20833) 
* First draft

* More improvements

* Add conversion script

* More improvements

* Add docs

* Address review

* Rename class to ConvEncoder

* Address review

* Apply suggestion

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update all DETR friends

* Add corresponding test

* Improve test

* Fix bug

* Add more tests

* Set out_features to last stage by default

Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MBP.localdomain>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2023-01-23 12:15:47 +01:00
Joao Gante
c8d719ff7e
Generate: precision fix in compute_transition_scores doctests (#21251) 2023-01-23 11:13:51 +00:00
Younes Belkada
e1cd78634a
[BLIP] fix doctest (#21217) 
* fix `blip` doctest

* Update src/transformers/models/blip/modeling_blip.py

Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>

Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
 2023-01-23 11:16:23 +01:00
Sylvain Gugger
4e730b3873
Skip failing test for now (#21226) 
skip failing test for now
 2023-01-20 20:46:11 -05:00
Younes Belkada
7fd902d335
[BLIP] fix docstring for BlipTextxxx (#21224) 
* fix `blip` docstring

* fix typo

* fix another typo
 2023-01-20 23:16:42 +01:00
Nicolas Patry
d54d7598bd
Microphone live inference catching up when inference is too slow (whisper). (#21219) 
* Microphone live inference catching up when inference is too slow
(whisper).

* Adding copyright.
 2023-01-20 21:33:43 +01:00
Sylvain Gugger
7fc1cb150c
Remove all hf-internal-testing checkpoints that can be removed (#21199) 
* Remove all hf-internal-testing checkpoints that can be removed

* Fix copies

* Put back processor_class in TF example

* Address review comment
 2023-01-20 13:19:58 -05:00
Steven Liu
142ad1a1cc
Fix task summary doctest (#21200) 
* add outputs to code snippets

* fix example text

* apply feedback

* style changes

* make style
 2023-01-20 09:58:07 -08:00
Jitesh Jain
425ff71c4e
Fix OneFormer Docstrings (#21215) 
* Fix processor

* Fix shape in docstring
 2023-01-20 17:37:11 +01:00
Yih-Dar
b0969cafd0
Make parallelism for CircleCI jobs work - but keep it 1 for now (#21157) 
* split tests

* test CI

* add if else

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-01-20 16:41:33 +01:00
Steven Liu
2553363826
Fix code example in training tutorial (#21201) 
change text to sentence
 2023-01-20 07:38:15 -08:00
Thomas Wang
7419d807ff
Declare __len__ method in PreTrainedTokenizerBase (#21210) 2023-01-20 15:54:33 +01:00
Yih-Dar
ef53017520
Fix GPTJ doctest (#21213) 
Replace the checkpoint - the current one has shape issue

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-01-20 15:35:00 +01:00
Yih-Dar
6ee6993fd9
Fix CONFIG_ARCHIVE_MAP_MAPPING_NAMES (#21207) 
fix typo + remove non-existent entry

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-01-20 15:22:10 +01:00
Yih-Dar
50540e18ff
Update huggingface_hub version (#21212) 
* update huggingface_hub version

* revert changes in setup.py

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-01-20 09:15:59 -05:00
Susnato Dhar
202d6863ce
deleted references of self.vocab_size and self.type_vocab_size for multiple models [TF implementation] (#21164) 2023-01-20 13:11:01 +00:00
Joao Gante
af37d183b3
Generate: documented function to compute the transition scores (#21191) 
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2023-01-20 12:50:01 +00:00
amyeroberts
91c2278b97
Update modeling doc strings FE -> IP (#21106) 
* Update docs examples FE -> IP

* Remove _IMAGE_PROCESSOR_FOR_DOC
 2023-01-20 11:18:10 +00:00
Arthur
5d3cb760a0
[Whispe] Fix pipeline after timestamp merges (#21198) 
* pass return_timestamps to pre-process

* add a test to test it

* test does not need device 0

* remove failing bit

* update test
 2023-01-20 10:31:40 +01:00
Nicolas Patry
5326460f14
Enabling live automatic-speech-recognition asr for Whisper. (#21196) 
* Enabling live `automatic-speech-recognition` asr for Whisper.

* Dummy change.
 2023-01-20 10:15:26 +01:00
Bartosz Szmelczynski
1b37fb5e17
Efficientformer (#20459) 
- Adds EfficientFormer V1 to transformers
- PR co-authored by @novice03  and @Bearnardd 

Co-authored-by: novice <pranavpulijala@gmail.com>
Co-authored-by: novice <44259234+novice03@users.noreply.github.com>
 2023-01-20 11:35:42 +03:00
Sylvain Gugger
862888a358
Add disclaimer for necessary fake models (#21178) 
* Add disclaimer for necessary fake models

* Address review comments

* Use for GPT-NeoX as well
 2023-01-19 14:16:15 -05:00