riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-31 02:02:21 +06:00
Code Issues Actions 33 Packages Projects Releases Wiki Activity
8,996 Commits 887 Branches 232 Tags 2.1 GiB
3a4376d008
Commit Graph

8996 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Patrick von Platen
3a4376d008
[Wav2Vec2ProcessorWithLM] Fix auto processor with lm (#15683) 2022-02-16 17:33:33 +01:00
Sylvain Gugger
cdc51ffd27
Add register method to AutoProcessor (#15669) 
* Add push_to_hub method to processors

* Fix test

* The other one too!

* Add register method to AutoProcessor

* Update src/transformers/models/auto/processing_auto.py

Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>

Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
 2022-02-16 09:13:33 -05:00
Eliott C
bc3379e12c
🔥 Remove build_doc_test github action (#15680) 2022-02-16 14:06:26 +01:00
Yih-Dar
d4692ad161
Fix dec_attn_mask in TFTransfoXLMainLayer (#15665) 
* fix attn

* clean-up

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-02-16 11:53:26 +00:00
Francesco Saverio Zuppichini
b87c044c79
Usage examples for logger (#15657) 
* logger

* Update docs/source/main_classes/logging.mdx

Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>

* Update docs/source/main_classes/logging.mdx

Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>

Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
 2022-02-16 10:15:13 +01:00
Sylvain Gugger
2d02f7b29b
Add push_to_hub method to processors (#15668) 
* Add push_to_hub method to processors

* Fix test

* The other one too!
 2022-02-15 21:14:04 -05:00
Stas Bekman
bee361c6f1
[t5/t0/mt5 models] faster/leaner custom layer norm (#14656) 
* [t5] faster/leaner custom layer norm

* wip

* apex.normalization.FusedRMSNorm

* cleanup

* cleanup

* add doc

* add catch all

* Trigger CI

* expand
 2022-02-15 16:49:57 -08:00
Santiago Castro
e3d1a8dabc
Add a missing space in a deprecation message (#15651) 2022-02-15 19:12:30 -05:00
Lysandre Debut
1ddf3c2b74
Fix vit test (#15671) 2022-02-15 18:55:38 -05:00
Lysandre Debut
943e2aa036
Fix model equivalence tests (#15670) 
* Fix model equivalence tests

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2022-02-15 18:55:22 -05:00
Yih-Dar
1690319217
Fix TFSequenceSummary's activation (#15643) 
* fix TFSequenceSummary

* fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-02-15 19:15:42 +00:00
Stas Bekman
faf4ff5974
[pipeline doc] fix api (#15660) 
* [pipeline doc] fix api

* remove duplicate
 2022-02-15 10:13:08 -08:00
Patrick von Platen
2e12b907ae
TF generate refactor - Greedy Search (#15562) 
* TF generate start refactor

* Add tf tests for sample generate

* re-organize

* boom boom

* Apply suggestions from code review

* re-add

* add all code

* make random greedy pass

* make encoder-decoder random work

* further improvements

* delete bogus file

* make gpt2 and t5 tests work

* finish logits tests

* correct logits processors

* correct past / encoder_outputs drama

* refactor some methods

* another fix

* refactor shape_list

* fix more shape list

* import shape
_list

* finish docs

* fix imports

* make style

* correct tf utils

* Fix TFRag as well

* Apply Lysandre's and Sylvais suggestions

* Update tests/test_generation_tf_logits_process.py

Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>

* Update src/transformers/tf_utils.py

Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>

* remove cpu according to gante

* correct logit processor

Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>
 2022-02-15 17:54:43 +01:00
Nicolas Patry
a3dbbc3467
Add decoder_kwargs to send to LM on asr pipeline. (#15646) 
Co-authored-by: Giuseppe Attanasio <giuseppeattanasio6@gmail.com>

Co-authored-by: Giuseppe Attanasio <giuseppeattanasio6@gmail.com>
 2022-02-15 17:53:24 +01:00
Nicolas Patry
cdf19c501d
Re-export KeyDataset. (#15645) 
* Re-export `KeyDataset`.

* Update the docs locations.
 2022-02-15 17:49:38 +01:00
Stas Bekman
28e6155d8a
add a network debug script and document it (#15652) 
* add a network debug script and document it

* doc
 2022-02-15 08:48:00 -08:00
Sylvain Gugger
5d8be090e0 Fix quality 2022-02-15 11:32:26 -05:00
Patrick von Platen
f45ac11fb3
Add section about doc testing (#15659) 
* Add doctesting section

* Improve

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2022-02-15 16:56:31 +01:00
Shamane Siri
80f1a59168
updated with latest PL and Ray (#15653) 2022-02-15 16:53:05 +01:00
Ngo Quang Huy
7bc4a01cb5
Update bad_words_ids usage (#15641) 
* Improve the parameter `bad_word_ids' usage

* Update the bad_words_ids strategy
 2022-02-15 16:44:34 +01:00
arampacha
67047b86ce
add scores to Wav2Vec2WithLMOutput (#15413) 
* add scores to Wav2Vec2WithLMOutput

* style fixup
 2022-02-15 16:40:50 +01:00
Sylvain Gugger
45f56580a7
Allow custom code for Processors (#15649) 
* Allow custom code for Processors

* Add more test

* Test all auto_map configs are properly set
 2022-02-15 09:44:35 -05:00
jonrbates
86a7845c0c
Fix typo in speech2text2 doc (#15617) 
Forward looks for inputs, not input_ids
 2022-02-15 13:54:34 +01:00
Javier de la Rosa
9eb7e9ba1d
Fix ASR pipelines from local directories with wav2vec models that have language models attached (#15590) 
* Fix loading pipelines with wav2vec models with lm when in local paths

* Adding tests

* Fix test

* Adding tests

* Flake8 fixes

* Removing conflict files :(

* Adding task type to test

* Remove unnecessary test and imports
 2022-02-15 13:45:08 +01:00
Alex Hedges
e1cbc073bf
Require tokenizers>=0.11.1 (#15266) 
`tokenizers` version that supports the feature to choose the direction of truncation
 2022-02-15 11:46:12 +01:00
fra
05a8580964 Revert "logger doc" 
This reverts commit 41168a49ce.
 2022-02-15 10:46:45 +01:00
fra
41168a49ce logger doc 2022-02-15 10:03:28 +01:00
Patrick von Platen
041fdc4a7e
[SpeechEncoderDecoder] Make sure no EOS is generated in test (#15655) 2022-02-15 09:13:55 +01:00
muzhi1991
e314c19a3f
fix bug for the log of RNG states are not properly loaded exception. (#15638) 
Co-authored-by: muz <muzhi1991@limuzhideMBP-2.lan>
 2022-02-14 20:30:55 -05:00
Sylvain Gugger
2e11a04337
Register feature extractor (#15634) 
* Rework AutoFeatureExtractor.from_pretrained internal

* Custom feature extractor

* Add more tests

* Add support for custom feature extractor code

* Clean up

* Add register API to AutoFeatureExtractor
 2022-02-14 13:35:16 -05:00
lewtun
0f71c29053
Remove redundant error logging in from_pretrained() method (#15631) 
* Remove error logging in from_pretrained() method
 2022-02-14 18:03:07 +01:00
NielsRogge
b090b79022
Make Swin work with VisionEncoderDecoderModel (#15527) 
* Add attribute_map

* Add mention in docs

* Set hidden_size attribute correctly

* Add note about Transformer-based models only

Co-authored-by: Niels Rogge <nielsrogge@Nielss-MBP.localdomain>
 2022-02-14 17:33:35 +01:00
Toni Kukurin
ec15da2445
Report only the failed imports in requires_backends (#15636) 2022-02-14 10:35:20 -05:00
Zhen Wang
2b8599b2df
Fix a bug that ignores max_seq_len in preprocess (#15238) 2022-02-14 13:18:40 +01:00
Yih-Dar
f52746d004
[Fix doc example] FlaxVisionEncoderDecoder (#15626) 
* Fix wrong checkpoint name: vit

* Fix missing import

* Fix more missing import

* make style

* Apply suggestions from code review

Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
 2022-02-14 12:48:23 +01:00
Sylvain Gugger
52d2e6f6e9
Add push to hub to feature extractor (#15632) 
* Add push to hub to feature extractor

* Quality

* Clean up
 2022-02-11 17:14:01 -05:00
Daniel Erenrich
4f403ea899
Fix grammar in tokenizer_summary (#15614) 
"to make ensure" is redundant.
 2022-02-11 16:51:30 -05:00
Sylvain Gugger
7a32e4722f
Custom feature extractor (#15630) 
* Rework AutoFeatureExtractor.from_pretrained internal

* Custom feature extractor

* Add more tests

* Add support for custom feature extractor code

* Clean up
 2022-02-11 16:43:54 -05:00
Stas Bekman
fcb0f74397
[research_projects] deal with security alerts (#15594) 
* [research_projects] deal with security alerts

* add a note of the original PL ver and warning
 2022-02-11 14:31:09 -05:00
Stas Bekman
f15c99fabf
[deepspeed docs] misc additions (#15585) 
* [deepspeed docs] round_robin_gradients

* training and/or eval/predict loss is

* Update docs/source/main_classes/deepspeed.mdx

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2022-02-11 10:54:04 -08:00
Sylvain Gugger
2dce350b33
Fix _configuration_file argument getting passed to model (#15629) 2022-02-11 13:46:08 -05:00
Steven Liu
85aee09e9a
🖍 remove broken link (#15615) 2022-02-11 12:33:55 -06:00
Joao Gante
2f40c728c9
TF MT5 embeddings resize (#15567) 
* Fix TF MT5 vocab resize

* more assertive testing
 2022-02-11 17:35:10 +00:00
Mishig Davaadorj
8c03df1010
Rebase (#15606) 2022-02-11 12:02:02 -05:00
Joao Gante
3fae83d23a
TF: Add informative warning for inexistent CPU backprop ops (#15612) 
* Add informative warning
 2022-02-11 16:16:26 +00:00
lewtun
7e4844fc2a
Enable ONNX export when PyTorch and TensorFlow installed in the same environment (#15625) 2022-02-11 16:25:06 +01:00
Sylvain Gugger
6cf06d198c
Mark "code in the Hub" API as experimental (#15624) 2022-02-11 09:55:31 -05:00
Patrick von Platen
45c7b5b1c7
[Generate] Small refactor (#15611) 2022-02-10 18:29:27 +01:00
Ngo Quang Huy
c0864d98ba
Correct JSON format (#15600) 2022-02-10 09:02:03 -08:00
lewtun
2e8b85f72e
Add local and TensorFlow ONNX export examples to docs (#15604) 
* Add local and TensorFlow ONNX export examples to docs

* Use PyTorch - TensorFlow split
 2022-02-10 16:31:00 +01:00