Matt
e9d9982e7c
Add TF protein notebook to notebooks doc ( #20271 )
2022-11-16 16:08:51 +00:00
Nicolas Patry
5ca479d252
Adding doctest for text-generation
pipeline. ( #20264 )
2022-11-16 16:57:46 +01:00
Nicolas Patry
449f2ae459
Adding doctest for text2text-generation
pipeline. ( #20261 )
2022-11-16 16:57:08 +01:00
Nicolas Patry
f6490180eb
Adding doctest for image-segmentation
pipeline. ( #20256 )
...
* Adding doctest for `image-segmentation` pipeline.
* Fixup.
2022-11-16 16:56:54 +01:00
Nicolas Patry
c389d35a7f
Adding a doctest for table-question-answering
pipeline. ( #20260 )
2022-11-16 16:45:42 +01:00
Jiahao Li
9681f052a1
Fix result saving errors of pytorch examples ( #20276 )
2022-11-16 09:51:04 -05:00
Mishig
e627e9b5ae
Complete doc migration ( #20267 )
2022-11-16 08:43:37 -05:00
Nicolas Patry
4fb34de99e
Adding an example for depth-estimation
pipeline. ( #20237 )
...
* Adding an example for `depth-estimation` pipeline.
* Adding missing internal link to tutorial.
2022-11-16 09:52:45 +01:00
Nicolas Patry
1f029b6ae7
Adding doctest for document-question-answering ( #20239 )
...
* Adding doctest for doc qa.
* Adding doctest for doc qa.
* Fixup.
2022-11-16 09:52:35 +01:00
Nicolas Patry
443aaaa1a7
Adding ASR pipeline example. ( #20226 )
...
* Adding ASR pipeline example.
* De indent.
* Example deindent.
* Fixing example ?
* Putting the example in a more prominent place.
* Fixup.
* Adding the file.
* Adding the doctest to the daily test.
* Fixing comments.
* transcriber name.
* Adding `>>>`.
* Removing assert.
2022-11-16 09:51:45 +01:00
Nicolas Patry
e434627858
Adding doctest for feature-extraction
. ( #20240 )
...
* Adding doctest for `feature-extraction`.
* Update feature_extraction.py
2022-11-16 09:51:31 +01:00
Nicolas Patry
529037fda5
Adding doctest for fill-mask
pipeline. ( #20241 )
2022-11-16 09:51:20 +01:00
Nicolas Patry
5e080c11bf
Updating the doctest for conversational. ( #20236 )
...
* Updating the doctest for conversational.
- Make it tested against
- Add explicit output in the test.
* Removing assert.
* Adding missing link.
2022-11-16 09:51:12 +01:00
Nicolas Patry
860ea8a574
Adding audio-classification
example in the doc. ( #20235 )
...
* Adding `audio-classification` example in the doc.
* Adding `>>>` to get the real test.
* Removing assert.
* Fixup.
2022-11-16 09:51:03 +01:00
Alara Dirik
a00b7e85ea
Adds image-guided object detection support to OWL-ViT ( #20136 )
...
Adds image-guided object detection method to OwlViTForObjectDetection class as described in the original paper. One-shot/ image-guided object detection enables users to use a query image to search for similar objects in the input image.
Co-Authored-By: Dhruv Karan k4r4n.dhruv@gmail.com
2022-11-16 09:07:46 +03:00
Yih-Dar
0d0d77693f
Allow trainer to return eval. loss for CLIP-like models ( #20214 )
...
* Allow trainer to return loss for CLIP-like models
* Apply suggestions
* update
* update
* update
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-11-15 19:47:10 +01:00
Zachary Mueller
822ae69c1b
Update reqs to include min gather_for_metrics Accelerate version ( #20242 )
...
* Update reqs to include min gather_for_metrics Accelerate version
* Other reqs
2022-11-15 13:28:00 -05:00
Ambuj Pawar
c19aa7acce
Add clip resources to the transformers documentation ( #20190 )
...
* WIP: Added CLIP resources from HuggingFace blog
* ADD: Notebooks documentation to clip
* Add link straight to notebook
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Change notebook links to colab
Co-authored-by: Ambuj Pawar <your_email@abc.example>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
2022-11-15 13:26:46 -05:00
Saad Mahmud
5b62f8ea2b
Add to DeBERTa resources ( #20155 )
...
* Add to DeBERTa resources
* Fix mistakes with chapter number
* Add fill-mask pipeline
* Add sequence, token and QA pipeline
* Change token classification pipeline order
* Remove flax script and notebook links
2022-11-15 13:26:07 -05:00
Matt
26ec7928d0
Slightly alter Keras dummy loss ( #20232 )
...
* Slightly alter Keras dummy loss
* Slightly alter Keras dummy loss
* Add sample weight to test_keras_fit
* Fix test_keras_fit for datasets
* Skip the sample_weight stuff for models where the model tester has no batch_size
2022-11-15 16:58:43 +00:00
Suraj Patil
7f74433814
[CLIP] allow loading projection layer in vision and text model ( #18962 )
...
* allow loading projection in text and vision model
* begin tests
* finish test for CLIPTextModelTest
* style
* add slow tests
* add new classes for projection heads
* remove with_projection
* add in init
* add in doc
* fix tests
* fix some more tests
* fix copies
* fix docs
* remove leftover from fix-copies
* add the head models in IGNORE_NON_AUTO_CONFIGURED
* fix docstr
* fix tests
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* add docstr for models
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2022-11-15 17:50:07 +01:00
Sylvain Gugger
9643ecf8ca
Enable PyTorch 1.13 ( #20168 )
...
* Try PT1.13 by removing torch scatter
* Skip failing tests
* Style
* Remvoe testing extras for repo utils
* Try with all decorators
* Try to wipe the cache
* Fix all tests?
* Try this way
* Fix comma
* Update to main
* Try with less deps
* Quality
2022-11-15 11:33:09 -05:00
Muhammad Sakib Khan Inan
777b1bfe62
New logging support to "Trainer" Class (ClearML Logger) ( #20184 )
...
* Init Update
* ClearML Callbacks integration
* update corrections
* args reporting updated
* {'tensorboard': False, 'pytorch': False}
* ClearML Tests added
* add clearml
* output_uri=True in Task.init
* reformatted integrations.py
* reformatted and fixed
* IF-ELSE statement issue on "has_clearml" resolved
* Add clearml in main callback docs
* Add additional clearml documentation
* Update src/transformers/integrations.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Accept suggestion
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Accept suggestion
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Small change in comments
* Make style clearml
* Accept suggestion
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Victor Sonck <victor.sonck@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2022-11-15 10:08:59 -05:00
NielsRogge
b4997382da
Fix MaskformerFeatureExtractor ( #20100 )
...
* Fix bug
* Add another fix
* Add print statement
* Apply fix
* Fix feature extractor
* Fix feature extractor
* Add print statements
* Add print statements
* Remove print statements
* Add instance segmentation integration test
* Add integration test for semantic segmentation
* Add draft for panoptic segmentation integration test
* Fix integration test for panoptic segmentation
* Remove slow annotator
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
2022-11-15 16:00:37 +01:00
TilmannR
6e3b014471
Fix docstring of CLIPTokenizer(Fast) ( #20233 )
2022-11-15 10:00:16 -05:00
Yih-Dar
cf7b98b807
Fix run_clip.py
( #20234 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-11-15 15:45:21 +01:00
Kendall
683cbc4c34
fixed spelling error in testing.mdx ( #20220 )
2022-11-15 09:40:06 -05:00
Yih-Dar
6ed6ed29b1
fix device issue ( #20227 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-11-15 15:21:16 +01:00
Matt
d3d5fa3e85
Add missing ESM autoclass ( #20177 )
...
* Add missing ESM autoclass
* Correct ESMFold checkpoint
2022-11-15 14:20:22 +00:00
Arthur
92cfe8b074
Remove authorized_missing_keys
in favor of _keys_to_ignore_on_load_missing ( #20228 )
2022-11-15 15:12:41 +01:00
Yong woo Song
2d92001076
Typo on doctring in ElectraTokenizer ( #20192 )
...
* chore: typo on docstring in tokenization_electra
* chore: typo on docstring in tokenization_electra
* update for check copies
2022-11-15 09:10:20 -05:00
amyeroberts
4c7e8d0900
Add object detection + segmentation transforms ( #20003 )
...
* Add transforms for object detection
* Update src/transformers/image_transforms.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Better var names & docstring
* Remove unused var desc in docstring
* Update src/transformers/image_transforms.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2022-11-15 12:50:03 +00:00
Younes Belkada
163ac3d3ee
Add Switch transformers ( #19323 )
...
* first commit
* add more comments
* add router v1
* clean up
- remove `tf` modeling files
* clean up
- remove `tf` modeling files
* clean up
* v0 routers
* added more router
- Implemented `ExpertsChooseMaskedRouter`
- added tests
- 2 more routers to implement
* last router
* improved docstring
- completed the docstring in `router.py`
- added more args in the config
* v0 sparse mlp
* replace wrong naming
* forward pass run
* update MOE layer
* small router update
* fixup
* consistency
* remove scatter router
* remove abstract layer
* update test and model for integration testing
* v1 conversion
* update
* hardcode hack
* all keys match
* add gin conversion, without additional libraries
* update conversion sctipy
* delete router file
* update tests wrt router deletion
* fix router issues
* update expert code
* update, logits match, code needsREFACTORING
* Refactor code
Co-authored-by: Younes Belkada <younesbelkada@users.noreply.github.com>
* add generate tests
Co-authored-by: younesbelkada <younesbelkada@gmail.com>
* add support for router loss
Co-authored-by: Younes Belkada <younesbelkada@users.noreply.github.com>
* fix forward error
* refactor a bit
* remove `FlaxSwitchTransformers` modules
* more tests pass
* Update code
Co-authored-by: Younes Belkada <younesbelkada@users.noreply.github.com>
* fixup
* fix tests
* fix doc
* fix doc + tokenization
* fix tokenizer test
* fix test
* fix loss output
* update code for backward pass
* add loss support
* update documentation
* fix documentation, clean tokenizer
* more doc fix, cleanup example_switch
* fix failing test
* fix test
* fix test
* fix loss issue
* move layer
* update doc and fix router capacity usage
* fixup
* add sparse mlp index for documentation on hub
* fixup
* test sparse mix architecture
* Apply suggestions from code review
* Update docs/source/en/model_doc/switch_transformers.mdx
* fixup on update
* fix tests
* fix another test
* attempt fix
* Update src/transformers/models/switch_transformers/configuration_switch_transformers.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* Update src/transformers/models/switch_transformers/convert_switch_transformers_original_flax_checkpoint_to_pytorch.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* try
* all tests pass
* fix jitter noise
* Apply suggestions from code review
* doc tests pass
* Update src/transformers/models/switch_transformers/modeling_switch_transformers.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* Update src/transformers/models/switch_transformers/modeling_switch_transformers.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* remove assert
* change config order
* fix readme japanese
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* remove parallelizable tests + add one liners
* remove ONNX config
* fix nits
- add `T5Tokenizer` in auto mapping
- remove `Switch Transformers` from ONNX supported models
* remove `_get_router`
* remove asserts
* add check in test for `router_dtype`
* add `SwitchTransformersConfig` in `run_pipeline_test`
* Update tests/pipelines/test_pipelines_summarization.py
* add huge model conversion script
* fix slow tests
- add better casting for `Linear8bitLt`
- remove `torchscript` tests
* add make dir
* style on new script
* fix nits
- doctest
- remove `_keys_to_ignore_on_load_unexpected`
* Update src/transformers/models/switch_transformers/configuration_switch_transformers.py
* add google as authors
* fix year
* remove last `assert` statements
* standardize vertical spaces
* fix failing import
* fix another failing test
* Remove strange àuthorized_keys`
* removing todo and padding that is never used
Co-authored-by: Arthur Zucker <arthur.zucker@gmail.com>
Co-authored-by: ybelkada <younes@huggingface.co>
Co-authored-by: Younes Belkada <younesbelkada@users.noreply.github.com>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Arthur Zucker <arthur@huggingface.co>
2022-11-15 13:06:45 +01:00
amyeroberts
55ba31908a
Add param_name to size_dict logs & tidy ( #20205 )
2022-11-15 10:52:58 +00:00
Younes Belkada
f1e8c48c5e
Add accelerate
support for ViT
family ( #20174 )
...
* add `accelerate` support for `ViT` family
- add `_no_split_modules`
- manually cast to the right `dtype`: to change
* enable `float16` for `deit`
* fix `make fixup`
* add `slow` test for `fp16` inference
* another safety check
* Update src/transformers/models/deit/modeling_deit.py
2022-11-15 11:06:01 +01:00
Arthur
11b2e45ccc
[WHISPER] Update modeling tests ( #20162 )
...
* Update modeling tests
* update tokenization test
* typo
* nit
* fix expected attention outputs
* Apply suggestions from code review
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
* Update tests from review
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* remove problematics kwargs passed to the padding function
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-11-15 11:04:58 +01:00
Arthur
f60eec4003
update relative positional embedding ( #20203 )
...
* update relative positional embedding
* make fix copies
* add `use_cache` to list of arguments
* fixup
* 1line fucntion
* add `test_decoder_model_past_with_large_inputs_relative_pos_emb`
* add relative pos embedding test for more models
* style
2022-11-15 10:46:34 +01:00
Yih-Dar
f9909fbf85
Make ImageSegmentationPipelineTests
less flaky ( #20147 )
...
* Fix ImageSegmentationPipelineTests
* Use 0.9
* no zip
* links to show images
* links to show images
* rebase
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-11-15 09:14:55 +01:00
bofeng huang
9625924c60
Update tokenizer_summary.mdx ( #20135 )
2022-11-15 01:18:13 +01:00
Wonhyeong Seo
8fadfd5035
[docs] set overflowing image width to auto-scale ( #20197 )
...
* docs: fix: set overflowing image width to auto-scale
* docs: fix: new language Korean is also affected
* docs: fix: unnecessary line break in index page
2022-11-15 01:13:40 +01:00
Nicolas Patry
25c451e5a0
Adding chunking for whisper (all seq2seq actually). Very crude matching algorithm. ( #20104 )
...
* Very crude matching algorithm.
* Fixing tests.
* Removing comments
* Adding warning + fix short matches.
* Cleanup tests.
* Quality.
* Less noisy.
* Fixup.
2022-11-14 22:32:50 +01:00
Joao Gante
938cb04789
Generate: add Bloom fixes for contrastive search ( #20213 )
2022-11-14 18:34:11 +00:00
amyeroberts
fda125638f
Downgrade log warning -> info ( #20202 )
2022-11-14 17:56:52 +00:00
Ming Liu
36b063ed4f
Update README.md ( #20188 )
...
There is typo in the original hyperlink.
Below is the original version:
Based on the script [`run_translation_no_trainer.py`](https://github.com/huggingface/transformers/blob/main/examples/pytorch/translation/**run_translationn_no_trainer.py **).
2022-11-14 12:53:02 -05:00
Yih-Dar
536e60d2c7
mark test_save_load_fast_init_from_base
as is_flaky
( #20200 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-11-14 18:51:33 +01:00
Sanchit Gandhi
af1a7c8ca3
[Examples] Generalise Seq2Seq ASR to handle Whisper ( #19519 )
...
* merge conflicts
* bos and eos in datacollator
* (temp) hardcode removal of attention mask
* freeze encoder
* actually freeze encoder
* set max length / num beams according to gen kwargs
* (temp) fix tests
* don't pop attn mask
* override return attention mask config from Hub
* Hub configs updated 🤗
* final fixes
* update type annotations
* backward comp
2022-11-14 17:45:46 +00:00
Wonhyeong Seo
7ecb039176
feat: add i18n issue template ( #20199 )
...
Part of #20183
docs: add relevant labels to i18n issue template
fix: typo on completion count
2022-11-14 12:36:58 -05:00
Wonhyeong Seo
07d8d6e2f7
docs: translated index page to korean ( #20180 )
...
docs: i18n: first draft of index page
docs: fix: first revision of index page
docs: i18n: missed section - supported frameworks
docs: fix: second revision of index page
review by @ArthurZucker
refactor: remove untranslated files from korean
docs: fix: remove untranslated references from toctree.yml
feat: enable korean docs in gh actions
docs: feat: add in_translation page as placeholder
docs: bug: testing if internal toc need alphabet chars
docs: fix: custom english anchor for non-alphanumeric headings
review by @sgugger
docs: i18n: translate comments on install methods in _config.py
docs: refactor: more concise wording for translations
2022-11-14 12:09:21 -05:00
Arthur
c149d366bb
add _keys_to_ignore_on_load_unexpected = [r"pooler"] ( #20210 )
2022-11-14 18:05:19 +01:00
Younes Belkada
8dcf494ef1
[ROC_BERT] Make CI happy ( #20175 )
...
* fix slow test
* Update tests/models/roc_bert/test_modeling_roc_bert.py
2022-11-14 18:04:25 +01:00