Commit Graph

15053 Commits

Author SHA1 Message Date
Nicolas Patry
67316444b0
Hotifx remove tuple for git config image processor. (#21278) 2023-01-24 16:07:50 +01:00
Matt
071529bd54
Use return_tensors="np" instead of "tf" (#21266)
Return NP instead of TF tensors for our data loading pipeline
2023-01-24 13:37:49 +00:00
Younes Belkada
f0fc791298
[Doc] fix broken link (#21276)
fix broken link
2023-01-24 11:18:48 +01:00
Yih-Dar
bde7378bf0
Skip test_multi_gpu_data_parallel_forward for UperNetModelTest (#21216)
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-01-24 10:41:16 +01:00
Sylvain Gugger
7119bb052a
v4.27.0.dev0 2023-01-23 16:52:35 -05:00
Sylvain Gugger
fd5cdaeea6
Models docstring (#21225)
* Clean all models

* Style

* Last to remove

* address review comments

* Address review comments
2023-01-23 14:33:18 -05:00
Maria Khalusova
9e86c4e193
Supported pipeline tasks update (#21268)
* added tasks from SUPPORTED_TASKS to docstrings

* make style

* sorted the tasks in the docstrtings in alphabetical order
2023-01-23 14:23:20 -05:00
Arthur
d8415ba42e
[Whisper] fix all issues with unk token (#21250)
* fix all issues with unk token

* fixup
2023-01-23 20:19:57 +01:00
amyeroberts
c18b4fbe9f
Add class properties with warnings (#21195)
* Replace reduce_labels with do_reduce_labels

* Replace only for __init__ and preprocess

* Add class properties with warnings

* Update tests
2023-01-23 18:45:27 +00:00
Arthur
b80b2218b5
[ci-daily] Fix pipeline tests (#21257)
* use streaming dataset

* fix whisper's test

* add rescale argument to chunk_iter
2023-01-23 19:32:49 +01:00
Maria Khalusova
275ad9d80a
Add: TensorFlow example for semantic segmentation task guide (#21223)
* wip: adding tf example for semantic segmentation guide

* completed the working example in tf

* make style

* Update docs/source/en/tasks/semantic_segmentation.mdx

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/en/tasks/semantic_segmentation.mdx

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* fixed a callback doc links

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
2023-01-23 13:32:15 -05:00
Maria Khalusova
2218dac5d2
Notebook examples grouping and update (#21265)
* Split the examples by modality, added missing examples

* fixed a link
2023-01-23 12:51:24 -05:00
amyeroberts
e2bd7f80d0
Update tests: replace feature extractor tests with image processor (#20768)
* Update imports and test fetcher

* Revert but keep test fetcher update

* Fix imports

* Fix all imports

* Replace fe with ip names

* Add generate kwargs to `AutomaticSpeechRecognitionPipeline` (#20952)

* Add generate kwargs to AutomaticSpeechRecognitionPipeline

* Add test for generation kwargs

* Update image processor parameters if creating with kwargs (#20866)

* Update parameters if creating with kwargs

* Shallow copy to prevent mutating input

* Pass all args in constructor dict - warnings in init

* Fix typo

* Rename tester class

* Rebase and tidy up

* Fixup

* Use ImageProcessingSavingTestMixin

* Update property ref in tests

* Update property ref in tests

* Update recently merged in models

* Small fix

Co-authored-by: bofeng huang <bofenghuang7@gmail.com>
2023-01-23 17:25:41 +00:00
amyeroberts
354ea44340
Replace reduce_labels with do_reduce_labels (#21218)
* Replace reduce_labels with do_reduce_labels

* Replace only for __init__ and preprocess

* Update tests
2023-01-23 17:21:33 +00:00
Joao Gante
1eda4a4102
Generate: save generation config with the models' .save_pretrained() (#21264) 2023-01-23 16:21:44 +00:00
amyeroberts
cf1a1eed70
Add missing checkpoint for doctest (#21258) 2023-01-23 15:27:25 +00:00
Mostafa Elhoushi
5603f78fc4
Add scikit-learn dependency to train langage-modeling (#21229) 2023-01-23 09:54:45 -05:00
Kambe Hiroyuki
929111698c
Add Japanese translation installation.mdx (#21241)
* Add Japanese translation installation.mdx

* Fixed for consistency with english version
2023-01-23 15:38:30 +01:00
Yih-Dar
cb6b56859a
Fix reformer CI (#21254)
* fix ReformerForSequenceClassification doc example

* fix ReformerForMaskedLM doc example

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-01-23 15:34:14 +01:00
raghavanone
eaace0c668
Optimize by not computing gradients for parameters set to requires_grad=False (#21236)
* Optimize by not computing gradients for parameters set to requires_grad=False

* Make change to retrigger the build

* Fix isort issue

* Fix issue
2023-01-23 09:27:59 -05:00
NielsRogge
6e4d3f0859
[GIT] Convert more checkpoints (#21245)
* Extend conversion script

* Remove print statement

Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
2023-01-23 15:19:27 +01:00
amyeroberts
66459ce319
Add test_image_processing_common.py (#20785)
* Add test_image_processing_common.py

* Fix typo

* Update imports and test fetcher

* Revert but keep test fetcher update

* Fix imports

* Fix all imports

* Formatting fix

* Update tests/test_image_processing_common.py
2023-01-23 13:48:30 +00:00
Ogundepo Odunayo
96b2b2de12
Extend Script to enable conversion of Encoder Only T5x Models to Pytorch (#20907)
* add converter for t5x_retrieval model

* update args

* Update src/transformers/models/t5/convert_t5x_checkpoint_to_pytorch.py

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* style  editing -> convert t5x to pytorch

* make style

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
2023-01-23 14:41:43 +01:00
NielsRogge
91ff7efeeb
[DETR and friends] Use AutoBackbone as alternative to timm (#20833)
* First draft

* More improvements

* Add conversion script

* More improvements

* Add docs

* Address review

* Rename class to ConvEncoder

* Address review

* Apply suggestion

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update all DETR friends

* Add corresponding test

* Improve test

* Fix bug

* Add more tests

* Set out_features to last stage by default

Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MBP.localdomain>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-01-23 12:15:47 +01:00
Joao Gante
c8d719ff7e
Generate: precision fix in compute_transition_scores doctests (#21251) 2023-01-23 11:13:51 +00:00
Younes Belkada
e1cd78634a
[BLIP] fix doctest (#21217)
* fix `blip` doctest

* Update src/transformers/models/blip/modeling_blip.py

Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>

Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
2023-01-23 11:16:23 +01:00
Sylvain Gugger
4e730b3873
Skip failing test for now (#21226)
skip failing test for now
2023-01-20 20:46:11 -05:00
Younes Belkada
7fd902d335
[BLIP] fix docstring for BlipTextxxx (#21224)
* fix `blip` docstring

* fix typo

* fix another typo
2023-01-20 23:16:42 +01:00
Nicolas Patry
d54d7598bd
Microphone live inference catching up when inference is too slow (whisper). (#21219)
* Microphone live inference catching up when inference is too slow
(whisper).

* Adding copyright.
2023-01-20 21:33:43 +01:00
Sylvain Gugger
7fc1cb150c
Remove all hf-internal-testing checkpoints that can be removed (#21199)
* Remove all hf-internal-testing checkpoints that can be removed

* Fix copies

* Put back processor_class in TF example

* Address review comment
2023-01-20 13:19:58 -05:00
Steven Liu
142ad1a1cc
Fix task summary doctest (#21200)
* add outputs to code snippets

* fix example text

* apply feedback

* style changes

* make style
2023-01-20 09:58:07 -08:00
Jitesh Jain
425ff71c4e
Fix OneFormer Docstrings (#21215)
* Fix processor

* Fix shape in docstring
2023-01-20 17:37:11 +01:00
Yih-Dar
b0969cafd0
Make parallelism for CircleCI jobs work - but keep it 1 for now (#21157)
* split tests

* test CI

* add if else

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-01-20 16:41:33 +01:00
Steven Liu
2553363826
Fix code example in training tutorial (#21201)
change text to sentence
2023-01-20 07:38:15 -08:00
Thomas Wang
7419d807ff
Declare __len__ method in PreTrainedTokenizerBase (#21210) 2023-01-20 15:54:33 +01:00
Yih-Dar
ef53017520
Fix GPTJ doctest (#21213)
Replace the checkpoint - the current one has shape issue

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-01-20 15:35:00 +01:00
Yih-Dar
6ee6993fd9
Fix CONFIG_ARCHIVE_MAP_MAPPING_NAMES (#21207)
fix typo + remove non-existent entry

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-01-20 15:22:10 +01:00
Yih-Dar
50540e18ff
Update huggingface_hub version (#21212)
* update huggingface_hub version

* revert changes in setup.py

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-01-20 09:15:59 -05:00
Susnato Dhar
202d6863ce
deleted references of self.vocab_size and self.type_vocab_size for multiple models [TF implementation] (#21164) 2023-01-20 13:11:01 +00:00
Joao Gante
af37d183b3
Generate: documented function to compute the transition scores (#21191)
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2023-01-20 12:50:01 +00:00
amyeroberts
91c2278b97
Update modeling doc strings FE -> IP (#21106)
* Update docs examples FE -> IP

* Remove _IMAGE_PROCESSOR_FOR_DOC
2023-01-20 11:18:10 +00:00
Arthur
5d3cb760a0
[Whispe] Fix pipeline after timestamp merges (#21198)
* pass return_timestamps to pre-process

* add a test to test it

* test does not need device 0

* remove failing bit

* update test
2023-01-20 10:31:40 +01:00
Nicolas Patry
5326460f14
Enabling live automatic-speech-recognition asr for Whisper. (#21196)
* Enabling live `automatic-speech-recognition` asr for Whisper.

* Dummy change.
2023-01-20 10:15:26 +01:00
Bartosz Szmelczynski
1b37fb5e17
Efficientformer (#20459)
- Adds EfficientFormer V1 to transformers
- PR co-authored by @novice03  and @Bearnardd 

Co-authored-by: novice <pranavpulijala@gmail.com>
Co-authored-by: novice <44259234+novice03@users.noreply.github.com>
2023-01-20 11:35:42 +03:00
Sylvain Gugger
862888a358
Add disclaimer for necessary fake models (#21178)
* Add disclaimer for necessary fake models

* Address review comments

* Use for GPT-NeoX as well
2023-01-19 14:16:15 -05:00
Clémentine Fourrier
87208a05af
Graphormer model for Graph Classification (#20968)
* [FT] First commit for graphormer architecture.

The model has no tokenizer, as it uses a collator and preprocessing function for its input management.
Architecture to be tested against original one.
The arch might need to be changed to fit the checkpoint, but a revert to the original arch will make the code less nice to read.
TODO: doc

* [FIX] removed test model

* [FIX] import error

* [FIX] black and flake

* [DOC] added paper refs

* [FIX] [DOC]

* [FIX] black

* [DOC] Updated READMEs

* [FIX] Order of imports + rm Tokenizer calls

* [FIX] Moved assert in class to prevent doc build failure

* [FIX] make fix-copies

* [Doc] update from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* [FIX] Removed Graphormer from Sequence classification model list

* [DOC] Added HF copyright to Cython file

* [DOC] Fixed comments

* [FIX] typos in class doc + removed config classes.

Todo: update doc from paper definitions

* [FIX] Removed dependency to fairseq, and replaced all asserts with Exception management

* [FIX] Homogeneized initialization of weights to pretrained constructor

* [FIX] [CP] Updated multi_hop parameter to get same results as in original implementation

* [DOC] Relevant parameter description in the configuration file

* [DOC] Updated doc and comments in main graphormer file

* [FIX] make style and quality checks

* [DOC] Fix doc format

* [FIX] [WIP] Updated part of the tests, though still a wip

* [FIX] [WIP]

* [FIX] repo consistency

* [FIX] Changed input names for more understandability

* [FIX] [BUG] updated num_classes params for propagation in the model

* simplified collator

* [FIX] Updated tests to follow new naming pattern

* [TESTS] Updated test suite along with model

* |FIX] rm tokenizer import

* [DOC] add link to graphormerdoc

* Changed section in doc from text model to graph model

* Apply suggestions from code review

Spacing, inits

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* [DOC] Explain algos_graphormer functions

* Cython soft import protection

* Rm call to Callable in configuration graphormer

* [FIX] replaced asserts with Exceptions

* Add org to graphormer checkpoints

* Prefixed classes with Graphormer

* Management of init functions

* format

* fixes

* fix length file

* update indent

* relaunching ci

* Errors for missing cython imports

* fix style

* fix style doc

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-01-19 13:05:59 -05:00
ydshieh
758bd39e81 revert Copyright 2023 2023-01-19 18:23:59 +01:00
Kambe Hiroyuki
705e332b46
Add Japanese translation index.mdx (#21186)
* Add Japanese translation index.mdx

* Fix the year of the license

* Change the models list to Japanese
2023-01-19 17:53:28 +01:00
Joao Gante
cbaaa2f6ac
Flax dtype-dependent numerical masking (#21197) 2023-01-19 16:43:42 +00:00
Younes Belkada
0b86e330b1
[CVT] Fix module initialization issue (#21193)
fix cvt init
2023-01-19 17:36:38 +01:00