riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-31 02:02:21 +06:00
Code Issues Actions 33 Packages Projects Releases Wiki Activity
8,287 Commits 887 Branches 232 Tags 2.1 GiB
d0e96c6de6
Commit Graph

8287 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Jeff Rasley
d0e96c6de6
[deepspeed] Enable multiple test runs on single box, defer to DS_TEST_PORT if set (#14331) 
* defer to DS_TEST_PORT if set

* style

Co-authored-by: Stas Bekman <stas@stason.org>
 2021-11-08 12:40:29 -08:00
Sylvain Gugger
dfb00bf644
Expand dynamic supported objects to configs and tokenizers (#14296) 
* Dynamic configs

* Add config test

* Better tests

* Add tokenizer and test

* Add to from_config

* With save
 2021-11-08 15:28:25 -05:00
nbertagnolli
de635af3f1
Changed relative imports to absolute to allow convert_graph_to_onnx.py to run as a script. (#14325) 
* Changed relative imports to absolute to allow convert_graph_to_onnx.py to be run as a script

* isorted code
 2021-11-08 10:56:44 -05:00
Nicolas Patry
a3ded170e2
Fixing mutable default argument in pipeline. (#14316) 
* Fixing mutable default argument.

* XX.

* Revert "XX."

This reverts commit 61d4bb333f.
 2021-11-08 16:22:28 +01:00
Nicolas Patry
9b78b070ef
Fixing tests on master. (#14317) 
* Fixing tests on master.

* Better fix.

* Lxmert doesn't have feature extractor but is bimodal.
 2021-11-08 08:28:26 -05:00
Anton Lozhkov
df1f94eb4a
[TFWav2Vec2Model] Fix input shapes in TFWav2Vec2WeightNormConv1D (#14319) 
* Add paddings to input shapes

* Add padding comment
 2021-11-08 15:58:28 +03:00
Anton Lozhkov
e30078b544
[Tests] Update audio classification tests to support torch 1.10 (#14318) 2021-11-08 14:15:56 +03:00
Patrick von Platen
b48faae364
[Marian Conversion] Fix eos_token_id conversion in conversion script (#14320) 2021-11-08 11:42:34 +01:00
Junbum Lee
c016dbdbda
Fix execution PATH for PPLM Example (#14287) 2021-11-06 10:33:47 -04:00
NielsRogge
34307bb358
Fix tests (#14289) 2021-11-06 10:08:58 -04:00
Nicolas Patry
24b30d4d2f
Handle long answer needs to be updated. (#14279) 
`start_` and `end_` tensors now contain a batch_size at this point.
 2021-11-06 10:04:30 -04:00
Xing Han Lu
843c326ee1
Update dpr.rst (#14300) 2021-11-06 09:41:02 -04:00
Sylvain Gugger
08a5f57567
Add new LFS prune API (#14294) 2021-11-05 18:58:51 -04:00
Patrick von Platen
4be78c22c9
[Hubert Docs] Make sure example uses a fine-tuned model (#14291) 2021-11-05 14:09:57 +01:00
Sylvain Gugger
a14d62b0b1
Pin TF until tests are fixed (#14283) 
* Pin TF until tests are fixed

* Also pin TF CPU
 2021-11-04 21:15:42 -04:00
Matt
b90a48f654
Removing Keras version pinning (#14280) 
* Removing Keras version pinning

* make fixup
 2021-11-04 17:58:28 +00:00
Chang Wang
fd8136fa75
improve rewrite state_dict missing _metadata (#14276) 2021-11-04 10:13:23 -04:00
Nicolas Patry
d29baf69bb
Fixing mishandling of ignore_labels. (#14274) 
Fixes #14272
 2021-11-04 09:47:52 -04:00
Nicolas Patry
68427c9beb
Fixing slow pipeline tests (#14260) 
* Fiixng slow pipeline tests

* Remove the image-segmentaiton override.

* Fixing clamping only in training.

* Wav2vec2.

* Remove last mention of `no_grad`.

* Fixing copies.

* Rename.
 2021-11-04 09:49:55 +01:00
Sylvain Gugger
1a674ce679
Add more instructions to the release guide (#14263) 
* Add more instructions to the release guide

* Apply suggestions from code review

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Address review comment

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
 2021-11-03 17:45:41 -04:00
Sylvain Gugger
f0d6e952c0
Quality explain (#14264) 
* Start PR doc

* Cleanup the quality checks and document them

* Add reference in the contributing guide

* Apply suggestions from code review

Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>

* Rename file as per review suggestion

Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
 2021-11-03 17:43:19 -04:00
Sylvain Gugger
a1c15ea855
Pin Keras cause they messed their release (#14262) 
* Pin Keras cause they messed their release

* Put != instead of <

* Try this way

* Back to the beginning but more agressive
 2021-11-03 15:03:09 -04:00
Nicolas Patry
1149243184
Fixing typo in error message. (#14226) 2021-11-03 19:28:57 +01:00
Dan Shirron
2c8957feea
Fix of issue #13327: Wrong weight initialization for TF t5 model (#14241) 
* Fix of issue #13327: Wrong weight initialization for TF t5 model

* run black formatter

* fix typo

* remove my name tag from comments

Co-authored-by: Shirron <dan.shirron@intel.com>
 2021-11-03 16:20:48 +00:00
Nicolas Patry
dec759e7e8
Adding support for truncation parameter on feature-extraction pipeline. (#14193) 
* Adding support for `truncation` parameter on `feature-extraction`
pipeline.

Fixes #14183

* Fixing tests on ibert, longformer, and roberta.

* Rebase fix.
 2021-11-03 15:48:00 +01:00
Dean Wyatte
27b1516d32
minimal fixes to run DataCollatorForWholeWordMask with return_tensors="np" and return_tensors="tf" (#13891) 
* minimal fixes to run DataCollatorForWholeWordMask with return_tensors="np" and return_tensors="tf"

* more consinstent implementation for numpy_mask_tokens
 2021-11-03 10:36:41 -04:00
Mishig Davaadorj
671569ddf7
Put load_image function in image_utils.py & fix image rotation issue (#14062) 
* Fix img load rotation

* Add `load_image` to `image_utils.py`

* Implement LoadImageTester

* Use hf-internal-testing dataset

* Add img utils comments

* Refactor LoadImageTester

* Import load_image under is_vision_available
 2021-11-03 14:53:05 +01:00
Patrick von Platen
89766b3d44
up (#14258) 2021-11-03 11:31:40 +01:00
Yih-Dar
bd21ed4099
Add cross attentions to TFGPT2Model (#14038) 
* Add cross attentions to TFGPT2Model

* change to is_pt_tf_cross_test

* A minor correction to a comment

* Remove n_ctx when creating self.crossattention

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2021-11-03 09:54:34 +01:00
NielsRogge
5f789a687a
Add LayoutXLMProcessor (and LayoutXLMTokenizer, LayoutXLMTokenizerFast) (#14115) 
* Add LayoutXLMTokenizer and LayoutXLMTokenizerFast

* Fix styling issues

* Fix more styling issues

* Fix more styling issues

* Fix docstring

* Fix unit tests

* Fix docs

* Fix unit tests

* Fix typos and styling issues

* Fix styling issues

* Fix docstring

* Make all tests of test_tokenization_layoutxlm pass

* Add LayoutXLMProcessor

* Make fixup

* Make all LayoutXLMProcessor tests pass

* Minor fixes

* Leave LayoutLMv2Processor tests unchanged

* Fix code quality

* Move LayoutXLM tokenizers and processor to separate folder

* Fix code quality

* Apply suggestions from code review

* Replace assertions by value errors

* Remove methods from fast tokenizer

Co-authored-by: King Yiu Suen <kingyiusuen@gmail.com>
 2021-11-03 08:59:44 +01:00
Sylvain Gugger
558f8543ba
Update Transformers to huggingface_hub >= 0.1.0 (#14251) 
* Update Transformers to huggingface_hub >= 0.1.0

* Forgot to save...

* Style

* Fix test
 2021-11-02 18:58:42 -04:00
lumliolum
519a677e87
Added Beit model output class (#14133) 
* add Beit model ouput class

* inherting from BaseModelOuputWithPooling

* updated docs if use_mean_pooling is False

* added beit specific outputs in model docs

* changed the import path

* Fix docs

Co-authored-by: Niels Rogge <niels.rogge1@gmail.com>
 2021-11-02 18:29:14 +01:00
Sylvain Gugger
bbaa3effbd
Fixes Beit training for PyTorch 1.10+ (#14249) 2021-11-02 13:07:20 -04:00
Sylvain Gugger
ad3e560bc7
Add PushToHubCallback in main init (#14246) 2021-11-02 12:15:15 -04:00
Anton Lozhkov
ce01122a3b
[Tests] Fix DistilHubert path (#14245) 
* Add audio-classification benchmarking results

* fix distilhubert path
 2021-11-02 17:53:50 +03:00
Yih-Dar
4a394cf53f
Fix test_configuration_tie in FlaxEncoderDecoderModelTest (#14076) 
* check test_configuration_tie

* Fix test_configuration_tie

* make test slow again

* Remove property and use model.module.bind

* revert to slow test

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2021-11-02 15:32:41 +05:30
Li-Huai (Allan) Lin
a767276fdd
Fix generation docstring (#14216) 
* Fix generation docstring

* Style
 2021-11-02 09:22:45 +01:00
NielsRogge
e20faa6f03
Add BeitForSemanticSegmentation (#14096) 
* Add first draft

* Make forward pass work

* Improve conversion script

* Add notebook that checks if it works

* Add BeitForSemanticSegmentation to the tests

* More improvements

* Make BeitForSemanticSegmentation consistent with Segformer

* Small bug fix

* Add BeitForSemanticSegmentation to docs

* Make sure model doesn't output hidden states when the user doesn't want to

* Make it possible to convert the large model

* Fix issue

* Fix conversion script for large model

* Add auxiliary_head option to semantic segmentation model

* Apply suggestions from @sgugger's review

* Apply suggestions from code review

* Fix failing test

Co-authored-by: Lysandre <lysandre.debut@reseau.eseo.fr>
 2021-11-01 19:55:45 +01:00
Walter Martin
8b32578119
improving efficiency of mlflow metric logging (#14232) 
Signed-off-by: Walter Martin <wamartin@microsoft.com>
 2021-11-01 13:46:11 -04:00
Suraj Patil
ce91bf9a34
[GPTJ] enable common tests and few fixes (#14190) 
* enable common tests, small fixes

* don't tie word embeds

* don't ignore lm_head
 2021-11-01 22:38:52 +05:30
mathor
70d5711848
Fix a writing issue in the comments of trainer.py (#14202) 2021-11-01 09:24:03 -04:00
Prabhudatta Das
33fb98338e
Raising exceptions instead of using assertions for few models (#14219) 
* raising exceptions instead of using assertions for few models

* fixed formatting issues

* fixing copy inconsistencies
 2021-11-01 08:53:13 -04:00
Nicolas Patry
999540dfe0
Tensor location is already handled (#14224) 
in `base.py` not in subclasses.
 2021-11-01 08:42:27 -04:00
Nicolas Patry
323f28dce2
Fixing image-segmentation tests. (#14223) 2021-11-01 08:25:34 -04:00
NielsRogge
7396095af7
Update README of QA examples (#14172) 2021-11-01 12:52:22 +01:00
Yih-Dar
9450bfcc6c
Add more missing models to models/__init__.py (#14177) 
* Add missing models to models/__init__.py

* Fix issues previously undetected

* Add UniSpeechSatForPreTraining to all_model_classes

* fix unispeech sat

* fix

* Add check_model_list() to check_repo.py

* Remove _ignore_models = ["bort"]

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: patrickvonplaten <patrick.v.platen@gmail.com>
 2021-11-01 10:52:36 +00:00
Lysandre
9fc1951711 Docs for v4.12.2 2021-10-29 14:51:05 -04:00
Lysandre
513fa30a63 Docs for v4.12.1 2021-10-29 13:49:50 -04:00
Lysandre Debut
63d91f449c
Torch 1.10 (#14169) 
* Torch 1.10

* torch scatter for 1.10

* style

* Skip tests
ok
 2021-10-29 13:43:43 -04:00
Haram Lee
e823d8198a
Add a condition for checking labels (#14211) 2021-10-29 13:12:10 -04:00