Suraj Patil
75ae287aec
fix flax examples tests ( #14646 )
...
* make tensorboard optional
* update test_fetcher for flax examples
* make the tests slow
2021-12-07 00:34:27 +05:30
Suraj Patil
cbe6026536
fix flax example tests ( #14643 )
2021-12-06 23:14:37 +05:30
Jay Zhang
1ccc033c56
Update the example of exporting Bart + BeamSearch to ONNX module to resolve comments. ( #14310 )
...
* Update code to resolve comments left in previous PR.
* Add README.md file for this example.
* Update examples/onnx/pytorch/translation/README.md
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update examples/onnx/pytorch/translation/README.md
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update examples/onnx/pytorch/translation/README.md
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update README.md file to resolve comments.
* Add a section name.
* Update examples/onnx/pytorch/translation/README.md
Co-authored-by: Gary Miguel <garymm@garymm.org>
* Add more comments for _convert_past_list_to_tuple().
* Change the default file name to a consistent one.
* Fix a format issue.
* Update examples/onnx/pytorch/translation/README.md
Co-authored-by: Gary Miguel <garymm@garymm.org>
* Update examples/onnx/pytorch/translation/run_onnx_exporter.py
Co-authored-by: Gary Miguel <garymm@garymm.org>
* Update examples/onnx/pytorch/translation/README.md
Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>
* Change the folder to summarization and address some other coments.
* Update the torch version.
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
Co-authored-by: Gary Miguel <garymm@garymm.org>
Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>
2021-12-06 14:01:51 +01:00
Julien Chaumond
6cdc3a7844
[urls to hub] Replace outdated model tags with their now-canonical pipeline types ( #14617 )
...
* Replace outdated model tags with their now-canonical pipeline types
* spam the CI till it's green
2021-12-06 04:35:01 -05:00
Suraj Patil
c5bd732ac6
Add Flax example tests ( #14599 )
...
* add test for glue
* add tests for clm
* fix clm test
* add summrization tests
* more tests
* fix few tests
* add test for t5 mlm
* fix t5 mlm test
* fix tests for multi device
* cleanup
* ci job
* fix metric file name
* make t5 more robust
2021-12-06 10:48:58 +05:30
Kamal Raj
803a8cd18f
updated readme with proper arguments ( #14624 )
2021-12-05 22:12:51 -05:00
(Bill) Yuchen Lin
3977b58437
fix a typo ( #14626 )
2021-12-05 11:31:23 +05:30
Leandro von Werra
43f953cc2e
Add CodeParrot 🦜 codebase ( #14536 )
...
* add readme skeleton
* update readme
* add initialization script
* add deduplication script
* add codeparrot training script
* add code generation evaluation
* add validation loss script
* add requirements
* update readme
* tweak readme
* make style
* add highlights to readme
* add CLIs to scripts
* add tokenizer training script
* add docstring to constant length dataset
* fix defaults in arguments
* update readme with cli
* move image to hub
* tweaks of readme
* fix cli commands
* add author
* explain env variables
* fix formatting
* Update examples/research_projects/codeparrot/README.md
Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>
* Apply suggestions from code review
Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>
* replace generic with gpt2 tokenizer
Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>
2021-12-02 10:41:35 +01:00
Sylvain Gugger
4df7d05a87
Doc new front ( #14590 )
...
* Convert PretrainedConfig doc to Markdown
* Use syntax
* Add necessary doc files (#14496 )
* Doc fixes (#14499 )
* Fixes for the new front
* Convert DETR file for table
* Title is needed
* Simplify a bit
* Even simpler
* Remove imports
* Fix typo in toctree (#14516 )
* Fix checkpoints badge
* Update versions.yml format (#14517 )
* Doc new front github actions (#14512 )
* Doc new front github actions
* Fix docstring
* Fix feature extraction utils import (#14515 )
* Address Julien's comments
* Push to doc-builder
* Ready for merge
* Remove old build and deploy
* Doc misc fixes (#14583 )
* Rm versions.yml from doc
* Fix converting.rst
* Rm pretrained_models from toctree
* Fix index links (#14567 )
* Fix links in README
* Localized READMEs
* Fix copy script
* Fix find doc script
* Update README_ko.md
Co-authored-by: Julien Chaumond <julien@huggingface.co>
Co-authored-by: Julien Chaumond <julien@huggingface.co>
* Adapt build command to new CLI tools (#14578 )
* Fix typo
* Fix doc interlinks (#14589 )
* Convert PretrainedConfig doc to Markdown
* Use syntax
* Rm pattern <[a-z]+(.html).*>
* Rm huggingface.co/transformers/master
* Rm .html
* Rm .html from index.mdx
* Rm .html from model_summary.rst
* Update index.mdx rm html
* Update remove .html
* Fix inner doc links
* Fix interlink in preprocssing.rst
* Update pr_checks
Co-authored-by: Sylvain Gugger <sylvain.gugger@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Convert PretrainedConfig doc to Markdown
* Use syntax
* Add necessary doc files (#14496 )
* Doc fixes (#14499 )
* Fixes for the new front
* Convert DETR file for table
* Title is needed
* Simplify a bit
* Even simpler
* Remove imports
* Fix checkpoints badge
* Fix typo in toctree (#14516 )
* Update versions.yml format (#14517 )
* Doc new front github actions (#14512 )
* Doc new front github actions
* Fix docstring
* Fix feature extraction utils import (#14515 )
* Address Julien's comments
* Push to doc-builder
* Ready for merge
* Remove old build and deploy
* Doc misc fixes (#14583 )
* Rm versions.yml from doc
* Fix converting.rst
* Rm pretrained_models from toctree
* Fix index links (#14567 )
* Fix links in README
* Localized READMEs
* Fix copy script
* Fix find doc script
* Update README_ko.md
Co-authored-by: Julien Chaumond <julien@huggingface.co>
Co-authored-by: Julien Chaumond <julien@huggingface.co>
* Adapt build command to new CLI tools (#14578 )
* Fix typo
* Fix doc interlinks (#14589 )
* Convert PretrainedConfig doc to Markdown
* Use syntax
* Rm pattern <[a-z]+(.html).*>
* Rm huggingface.co/transformers/master
* Rm .html
* Rm .html from index.mdx
* Rm .html from model_summary.rst
* Update index.mdx rm html
* Update remove .html
* Fix inner doc links
* Fix interlink in preprocssing.rst
* Update pr_checks
Co-authored-by: Sylvain Gugger <sylvain.gugger@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Styling
Co-authored-by: Mishig Davaadorj <mishig.davaadorj@coloradocollege.edu>
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
Co-authored-by: Julien Chaumond <julien@huggingface.co>
2021-12-01 14:13:02 -05:00
Thomas Viehmann
6ed9882ddb
use functional interface for softmax in attention ( #14198 )
...
* use functional interface instead of instantiating module and immediately calling it
* fix torch.nn.functional to nn.functional. Thank you Stas!
2021-11-30 11:47:33 -05:00
Rahul Nadkarni
8332327dca
Fix sentinel token IDs in data collator for Flax T5 pretraining script ( #14477 )
2021-11-29 17:30:17 +01:00
Kamal Raj
2bd950ca47
[Flax] token-classification model steps enumerate start from 1 ( #14547 )
...
* step start from 1
* Updated cur_step calcualtion
2021-11-29 21:55:59 +05:30
Nicholas Broad
69e16abf98
Switch from using sum for flattening lists of lists in group_texts ( #14472 )
...
* remove sum for list flattening
* change to chain(*)
* make chain object a list
* delete empty lines
per sgugger's suggestions
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Nicholas Broad <nicholas@nmbroad.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2021-11-22 16:17:26 -05:00
Stas Bekman
11f65d4158
[test] add test for --config_overrides ( #14466 )
...
* add test for --config_overrides
* remove unneeded parts of the test
2021-11-22 11:33:43 -05:00
Shang Zhang
a59e7c1ed4
Add QDQBert model and quantization examples of SQUAD task ( #14066 )
...
* clean up branch for add-qdqbert-model
* README update for QAT example; update docstrings in modeling_qdqbert.py
* Update qdqbert.rst
* Update README.md
* Update README.md
* calibration data using traning set; QAT example runs in fp32
* re-use BERTtokenizer for qdqbert
* Update docs/source/model_doc/qdqbert.rst
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update docs/source/model_doc/qdqbert.rst
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update docs/source/model_doc/qdqbert.rst
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* remove qdqbert tokenizer
* Update qdqbert.rst
* update evaluate-hf-trt-qa.py
* update configuration_qdqbert.py
* update modeling_qdqbert.py: add copied statement; replace assert with ValueError
* update copied from statement
* add is_quantization_available; run make fix-copies
* unittest add require_quantization
* add backend dependency to qdqbert model
* update README; update evaluate script; make style
* lint
* docs qdqbert update
* circleci build_doc add pytorch-quantization for qdqbert
* update README
* update example readme with instructions to upgrade TensorRT to 8.2
* Update src/transformers/models/qdqbert/configuration_qdqbert.py
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
* Update src/transformers/models/qdqbert/configuration_qdqbert.py
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
* Update src/transformers/models/qdqbert/configuration_qdqbert.py
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
* Update src/transformers/models/qdqbert/configuration_qdqbert.py
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
* change quantization to pytorch_quantization for backend requirement
* feed_forward_chunking not supported in QDQBert
* make style
* update model docstrings and comments in testing scripts
* rename example to quantization-qdqbert; rename example scripts from qat to quant
* Update src/transformers/models/qdqbert/modeling_qdqbert.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* rm experimental functions in quant_trainer
* qa cleanup
* make fix-copies for docs index.rst
* fix doctree; use post_init() for qdqbert
* fix early device assignment for qdqbert
* fix CI:Model templates runner
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2021-11-19 13:33:39 -05:00
Patrick von Platen
efea0f868b
[Speech Recognition] More examples
...
Add more XLS-R training runs to the official examples
2021-11-18 23:42:02 +01:00
William Held
01f8e639d3
Recover Deleted XNLI Instructions ( #14437 )
2021-11-17 20:16:47 -05:00
Antonio Carlos Falcão Petri
7544efc92e
[Gradient checkpoining] Update Wav2Vec scripts ( #14036 )
...
Co-authored-by: Stas Bekman <stas@stason.org>
2021-11-17 18:37:21 +01:00
Eldar Kurtic
9fd937ead1
Replace BertLayerNorm with LayerNorm ( #14385 )
...
Running Movement pruning experiments with the newest HuggingFace would crash due to non-existing BertLayerNorm.
2021-11-15 13:25:10 -05:00
Matt
267867e851
Quick fix to TF summarization example ( #14401 )
2021-11-15 13:45:51 +00:00
Patrick von Platen
55f49c5f4b
[Wav2Vec2 Example] Improve fine-tuning script ( #14373 )
...
* improve some stuff
* finish
* correct last
2021-11-12 16:35:57 +01:00
Stas Bekman
77262ef750
fix --gradient_checkpointing ( #13964 )
2021-11-11 17:50:21 +01:00
Matt
7f20bf0d43
Fixing requirements for TF LM models and use correct model mappings ( #14372 )
...
* Fixing requirements for TF LM models and use correct model mappings
* make style
2021-11-11 15:34:00 +00:00
Suraj Patil
e92190c0f8
Fix Flax params dtype ( #13098 )
...
* fix inits
* fix embed dtype
* fix embed dtype
* add test to check default dtype
* quality
* add type conversion methods for flax models
* more robust casting
* cast sinusoidal positions
* update pegasus
* update albert
* update test
* make sure dtype is passed to every module
* style
* fix electra dense
* fix t5
* quality
* add more tests
* better name
* use the dtype for lm head computation
* fix albert
* style
* fix albert embed dtype
* more tests
* fix vision enc-dec
* cleanup
* fix embed dtype pegasus
* fix default param test
* doc
* update template
* fix final_logits_bias dtype
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* fix doc
* fix doc
* add detailed docstring for dtype parameter
* remove un-necessary import
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
2021-11-11 14:45:20 +05:30
Suraj Patil
85a4bda4f4
bump flax version ( #14343 )
2021-11-09 22:15:22 +05:30
karthikrangasai
4f24058c58
Update Seq2Seq QA example script to use SQuAD metric. ( #14335 )
...
* Update postporcessing accordingly to use SQuAD metric.
* Update assets accordingly based on SQuAD metrics.
* Fix function naming error.
2021-11-09 08:04:23 -05:00
Junbum Lee
c016dbdbda
Fix execution PATH for PPLM Example ( #14287 )
2021-11-06 10:33:47 -04:00
Sylvain Gugger
08a5f57567
Add new LFS prune API ( #14294 )
2021-11-05 18:58:51 -04:00
Sylvain Gugger
558f8543ba
Update Transformers to huggingface_hub >= 0.1.0 ( #14251 )
...
* Update Transformers to huggingface_hub >= 0.1.0
* Forgot to save...
* Style
* Fix test
2021-11-02 18:58:42 -04:00
NielsRogge
7396095af7
Update README of QA examples ( #14172 )
2021-11-01 12:52:22 +01:00
Thomas Wang
5b45422b58
Remove n_ctx from configs ( #14165 )
...
* Remove n_ctx from configs
* Fix GPTJ and OpenAIGPT, both are acceptable breaking changes as there are no configs such that it breaks
* Remove unecessary n_positions from TFOpenAIGPT
2021-10-29 11:50:25 +02:00
Patrick von Platen
ba71f1b57f
Update README.md
2021-10-28 19:43:05 +02:00
Lysandre
b8fad022a0
v4.13.0.dev0
2021-10-28 12:56:46 -04:00
Lysandre
62bf536631
Release v4.12.0
2021-10-28 12:09:49 -04:00
Anton Lozhkov
78b6a2ecbd
Add audio-classification benchmarking results ( #14192 )
2021-10-28 15:59:18 +03:00
Patrick von Platen
88cd82e801
Update README.md
2021-10-28 02:35:01 +02:00
Patrick von Platen
e118db15d6
Update README.md
2021-10-28 01:59:27 +02:00
Patrick von Platen
01b1466983
[TPU tests] Enable first TPU examples pytorch ( #14121 )
...
* up
* up
* fix
* up
* Update examples/pytorch/test_xla_examples.py
* correct labels
* up
* up
* up
* up
* up
* up
2021-10-28 01:22:28 +02:00
Emanuel Huber
ebd48c6de5
Replace assertions with ValueError exception ( #14142 )
...
Updated masked-language modeling examples in pytorch
with convention defined by #12789
2021-10-26 17:14:29 -04:00
Matthew Goldey
42bfb83d74
fix typos in error messages in speech recognition example and modelcard.py ( #14166 )
...
* specify the text column name in the error message
* pluralize the word fields
2021-10-26 16:36:26 -04:00
Jangwon Park
41dad89f70
chore: typo on ner accelerate example code ( #14150 )
2021-10-26 16:23:41 -04:00
Patrick von Platen
9799f4e150
Update README.md
2021-10-26 18:59:25 +02:00
Patrick von Platen
f5ed19f57d
[Speech Recognition] - Distributed training: Make sure vocab file removal and creation don't interfer ( #14161 )
...
* up
* better
2021-10-26 15:59:33 +02:00
Patrick von Platen
e248e9b042
up ( #14154 )
2021-10-26 13:08:18 +02:00
Patrick von Platen
c99a2832ed
Update README.md
2021-10-25 19:50:36 +02:00
Patrick von Platen
1a9381c60d
Update README.md
2021-10-25 19:49:51 +02:00
Reza Gharibi
2ac65551ea
Fix rendering of examples version links ( #14134 )
2021-10-25 07:45:44 -04:00
karthikrangasai
1b871e091b
Supporting Seq2Seq model for question answering task ( #13432 )
...
* Add seq2seq example for QnA on SQuAD Dataset.
* Changes from review - Fixing styling mistakes.
* Added how to example in README, simplified the access to dataset's preprocess function.
* Added tests for the seq2seq QA example.
* Change dataset column name to fix tests.
* Fix test command mistake.
* Add missing argument 'ignore_pad_token_for_loss' from DataTrainingArguments.
* Add missing argument 'num_beams' from DataTrainingArguments.
* Fix processing of output predicted token ids so that tokenizer decode gets appropriate input. Updated assertion conditions on the tests.
2021-10-25 07:42:53 -04:00
Antonio Carlos Falcão Petri
05a2afc252
Add missing --validation_split_percentage data args ( #14119 )
2021-10-22 19:04:54 +02:00
lee1jun
d432a654f6
fix typo in license docstring ( #14094 )
...
last line: "# limitations under the License." is missing
2021-10-21 15:31:32 -04:00
Anton Lozhkov
e03544a138
[Examples] Add audio classification notebooks ( #14099 )
...
* Update SEW integration test tolerance
* Add audio classification notebooks
2021-10-21 19:15:46 +03:00
Christopher Akiki
f9c16b02e3
Replace "Masked" with "Causal" in TF CLM example ( #14014 )
2021-10-21 16:19:30 +01:00
Patrick von Platen
e9d2a639f4
up ( #14093 )
2021-10-21 10:30:02 +02:00
Sylvain Gugger
f875fb0e5f
Fix label attribution in token classification examples ( #14055 )
2021-10-20 07:55:14 -04:00
Patrick von Platen
53dc39d821
up ( #14079 )
2021-10-20 13:01:42 +02:00
Patrick von Platen
0bc2e54f00
Add ASR colabs ( #14067 )
...
* up
* Update notebooks/README.md
2021-10-20 11:51:41 +02:00
Anton Lozhkov
dbaf49203e
[Examples] Use Audio feature in speech classification ( #14052 )
...
* Update SEW integration test tolerance
* Update audio classification
* Update test
* Remove torchaudio
* Add dataset revision
* Hub branch naming
* Revert dataset revisions
* Update datasets
2021-10-20 12:22:43 +03:00
Weizhe Yuan
7a3147e9b8
fix typo ( #14049 )
2021-10-18 18:03:11 -04:00
Patrick von Platen
bdf31d6e0a
[Speech] Move all examples to new audio feature ( #14045 )
...
* up
* up
* up
* finish
2021-10-18 12:52:40 +02:00
Patrick von Platen
37c5759cbe
[Speech Examples] Add new audio feature ( #14027 )
...
* finish
* up
* finish all
* up
2021-10-17 23:01:03 +02:00
jacksukk
d5b82bb70c
Fixed horizon_length for PPLM ( #13886 )
...
* fixed horizon_length
* fixed horizon_length
* fix style
2021-10-14 21:46:09 -04:00
Patrick von Platen
7fb2a8b3d9
up ( #14008 )
2021-10-14 15:46:22 +02:00
Sylvain Gugger
0ef61d392c
Revert "Skip faulty test"
...
This reverts commit 5b6bd4e788
2021-10-14 09:02:41 -04:00
Sylvain Gugger
5b6bd4e788
Skip faulty test
2021-10-13 22:04:40 -04:00
Patrick von Platen
d45fc7da3d
[Speech Examples] Add pytorch speech pretraining ( #13877 )
...
* adapt wav2vec2
* add example
* add files
* adapt
* remove bogus file
* Apply suggestions from code review
* adapt files more
* upload changes
* del old files
* up
* up
* up
* up
* up
* correct gradient checkpoitning
* add readme
* finish
* finish
* up
* more fixes
* up
* up
* add demo run to readme
* up
2021-10-12 00:46:32 +02:00
Chungman Lee
46dfe99e44
Fix typo in README.md ( #13883 )
2021-10-08 14:25:32 -04:00
Dhananjay Shettigar
319beb64eb
#12789 Replace assert statements with exceptions ( #13909 )
...
* #12789 Replace assert statements with exceptions
* fix-copies: made copy changes to utils_qa.py in examples/pytorch/question-answering and examples/tensorflow/question-answering
* minor refactor for clarity
2021-10-07 09:09:01 -04:00
Jay Zhang
279ce5b705
Add an example of exporting BartModel + BeamSearch to ONNX module. ( #13765 )
...
* Add all example files.
* Reformat files by black.
* Style.
* Remove unused imports.
Co-authored-by: Morgan Funtowicz <funtowiczmo@gmail.com>
2021-10-07 12:07:02 +02:00
Akul Agrawal
dac7798144
Update run_qa.py ( #13857 )
2021-10-05 23:10:24 -04:00
Md Saiful Islam Sayef
155b23008e
Update FSNER code in examples->research_projects->fsner ( #13864 )
...
* Add example use of few-shot named entity recognition model in research_projects folder.
* Apply suggestions from code review
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update fsner example README.md.
- Change wrong import FSNERTokenizerWrapper to FSNERTokenizerUtils in the example code
- Add a link to the model identifier
* Update examples/research_projects/fsner/src/fsner/model.py
Fix spelling mistake in the default parameter of pretrained model name.
Co-authored-by: Stefan Schweter <stefan@schweter.it>
* Add example use of few-shot named entity recognition model in research_projects folder.
* Apply suggestions from code review
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update fsner example README.md.
- Change wrong import FSNERTokenizerWrapper to FSNERTokenizerUtils in the example code
- Add a link to the model identifier
* Update examples/research_projects/fsner/src/fsner/model.py
Fix spelling mistake in the default parameter of pretrained model name.
Co-authored-by: Stefan Schweter <stefan@schweter.it>
* Run Checking/fixing examples/flax/language-modeling/run_clm_flax.py examples/flax/question-answering/run_qa.py examples/flax/question-answering/utils_qa.py examples/flax/token-classification/run_flax_ner.py examples/legacy/multiple_choice/utils_multiple_choice.py examples/legacy/seq2seq/seq2seq_trainer.py examples/legacy/token-classification/utils_ner.py examples/pytorch/image-classification/run_image_classification.py examples/pytorch/language-modeling/run_clm.py examples/pytorch/language-modeling/run_clm_no_trainer.py examples/pytorch/language-modeling/run_mlm.py examples/pytorch/language-modeling/run_mlm_no_trainer.py examples/pytorch/language-modeling/run_plm.py examples/pytorch/multiple-choice/run_swag.py examples/pytorch/multiple-choice/run_swag_no_trainer.py examples/pytorch/question-answering/run_qa.py examples/pytorch/question-answering/run_qa_beam_search.py examples/pytorch/question-answering/run_qa_beam_search_no_trainer.py examples/pytorch/question-answering/run_qa_no_trainer.py examples/pytorch/summarization/run_summarization.py examples/pytorch/summarization/run_summarization_no_trainer.py examples/pytorch/test_examples.py examples/pytorch/text-classification/run_glue.py examples/pytorch/text-classification/run_glue_no_trainer.py examples/pytorch/text-classification/run_xnli.py examples/pytorch/token-classification/run_ner.py examples/pytorch/token-classification/run_ner_no_trainer.py examples/pytorch/translation/run_translation.py examples/pytorch/translation/run_translation_no_trainer.py examples/research_projects/adversarial/utils_hans.py examples/research_projects/distillation/grouped_batch_sampler.py examples/research_projects/fsner/setup.py examples/research_projects/fsner/src/fsner/__init__.py examples/research_projects/fsner/src/fsner/model.py examples/research_projects/fsner/src/fsner/tokenizer_utils.py examples/research_projects/jax-projects/big_bird/evaluate.py examples/research_projects/jax-projects/hybrid_clip/run_hybrid_clip.py examples/tensorflow/language-modeling/run_clm.py examples/tensorflow/multiple-choice/run_swag.py examples/tensorflow/question-answering/run_qa.py examples/tensorflow/summarization/run_summarization.py examples/tensorflow/text-classification/run_glue.py examples/tensorflow/translation/run_translation.py src/transformers/__init__.py src/transformers/commands/add_new_model.py src/transformers/configuration_utils.py src/transformers/convert_slow_tokenizer.py src/transformers/data/__init__.py src/transformers/data/data_collator.py src/transformers/data/datasets/glue.py src/transformers/data/datasets/language_modeling.py src/transformers/data/datasets/squad.py src/transformers/deepspeed.py src/transformers/dependency_versions_table.py src/transformers/feature_extraction_sequence_utils.py src/transformers/file_utils.py src/transformers/generation_flax_utils.py src/transformers/generation_logits_process.py src/transformers/generation_tf_utils.py src/transformers/generation_utils.py src/transformers/integrations.py src/transformers/modelcard.py src/transformers/modeling_flax_utils.py src/transformers/modeling_outputs.py src/transformers/modeling_tf_utils.py src/transformers/modeling_utils.py src/transformers/models/__init__.py src/transformers/models/albert/__init__.py src/transformers/models/albert/modeling_albert.py src/transformers/models/albert/modeling_flax_albert.py src/transformers/models/albert/tokenization_albert_fast.py src/transformers/models/auto/__init__.py src/transformers/models/auto/auto_factory.py src/transformers/models/auto/configuration_auto.py src/transformers/models/auto/dynamic.py src/transformers/models/auto/feature_extraction_auto.py src/transformers/models/auto/modeling_auto.py src/transformers/models/auto/modeling_flax_auto.py src/transformers/models/auto/modeling_tf_auto.py src/transformers/models/auto/tokenization_auto.py src/transformers/models/bart/configuration_bart.py src/transformers/models/bart/modeling_bart.py src/transformers/models/bart/modeling_flax_bart.py src/transformers/models/bart/modeling_tf_bart.py src/transformers/models/barthez/tokenization_barthez_fast.py src/transformers/models/beit/__init__.py src/transformers/models/beit/configuration_beit.py src/transformers/models/beit/modeling_beit.py src/transformers/models/beit/modeling_flax_beit.py src/transformers/models/bert/configuration_bert.py src/transformers/models/bert/modeling_bert.py src/transformers/models/bert/modeling_flax_bert.py src/transformers/models/bert_generation/configuration_bert_generation.py src/transformers/models/bert_generation/modeling_bert_generation.py src/transformers/models/big_bird/configuration_big_bird.py src/transformers/models/big_bird/modeling_big_bird.py src/transformers/models/big_bird/modeling_flax_big_bird.py src/transformers/models/big_bird/tokenization_big_bird_fast.py src/transformers/models/bigbird_pegasus/configuration_bigbird_pegasus.py src/transformers/models/bigbird_pegasus/modeling_bigbird_pegasus.py src/transformers/models/blenderbot/configuration_blenderbot.py src/transformers/models/blenderbot/modeling_blenderbot.py src/transformers/models/blenderbot/modeling_tf_blenderbot.py src/transformers/models/blenderbot_small/configuration_blenderbot_small.py src/transformers/models/blenderbot_small/modeling_blenderbot_small.py src/transformers/models/blenderbot_small/modeling_tf_blenderbot_small.py src/transformers/models/byt5/tokenization_byt5.py src/transformers/models/camembert/tokenization_camembert_fast.py src/transformers/models/canine/configuration_canine.py src/transformers/models/canine/modeling_canine.py src/transformers/models/clip/configuration_clip.py src/transformers/models/clip/convert_clip_original_pytorch_to_hf.py src/transformers/models/clip/modeling_clip.py src/transformers/models/clip/modeling_flax_clip.py src/transformers/models/clip/tokenization_clip.py src/transformers/models/convbert/modeling_convbert.py src/transformers/models/ctrl/configuration_ctrl.py src/transformers/models/deberta/modeling_tf_deberta.py src/transformers/models/deberta_v2/__init__.py src/transformers/models/deberta_v2/modeling_deberta_v2.py src/transformers/models/deberta_v2/modeling_tf_deberta_v2.py src/transformers/models/deit/configuration_deit.py src/transformers/models/deit/modeling_deit.py src/transformers/models/detr/configuration_detr.py src/transformers/models/detr/modeling_detr.py src/transformers/models/distilbert/__init__.py src/transformers/models/distilbert/configuration_distilbert.py src/transformers/models/distilbert/modeling_distilbert.py src/transformers/models/distilbert/modeling_flax_distilbert.py src/transformers/models/dpr/configuration_dpr.py src/transformers/models/dpr/modeling_dpr.py src/transformers/models/electra/modeling_electra.py src/transformers/models/electra/modeling_flax_electra.py src/transformers/models/encoder_decoder/__init__.py src/transformers/models/encoder_decoder/modeling_encoder_decoder.py src/transformers/models/encoder_decoder/modeling_flax_encoder_decoder.py src/transformers/models/flaubert/configuration_flaubert.py src/transformers/models/flaubert/modeling_flaubert.py src/transformers/models/fnet/__init__.py src/transformers/models/fnet/configuration_fnet.py src/transformers/models/fnet/convert_fnet_original_flax_checkpoint_to_pytorch.py src/transformers/models/fnet/modeling_fnet.py src/transformers/models/fnet/tokenization_fnet.py src/transformers/models/fnet/tokenization_fnet_fast.py src/transformers/models/fsmt/configuration_fsmt.py src/transformers/models/fsmt/modeling_fsmt.py src/transformers/models/funnel/configuration_funnel.py src/transformers/models/gpt2/__init__.py src/transformers/models/gpt2/configuration_gpt2.py src/transformers/models/gpt2/modeling_flax_gpt2.py src/transformers/models/gpt2/modeling_gpt2.py src/transformers/models/gpt2/modeling_tf_gpt2.py src/transformers/models/gpt_neo/configuration_gpt_neo.py src/transformers/models/gpt_neo/modeling_gpt_neo.py src/transformers/models/gptj/__init__.py src/transformers/models/gptj/configuration_gptj.py src/transformers/models/gptj/modeling_gptj.py src/transformers/models/herbert/tokenization_herbert_fast.py src/transformers/models/hubert/__init__.py src/transformers/models/hubert/configuration_hubert.py src/transformers/models/hubert/convert_hubert_original_s3prl_checkpoint_to_pytorch.py src/transformers/models/hubert/modeling_hubert.py src/transformers/models/hubert/modeling_tf_hubert.py src/transformers/models/ibert/modeling_ibert.py src/transformers/models/layoutlm/__init__.py src/transformers/models/layoutlm/configuration_layoutlm.py src/transformers/models/layoutlm/modeling_layoutlm.py src/transformers/models/layoutlmv2/__init__.py src/transformers/models/layoutlmv2/configuration_layoutlmv2.py src/transformers/models/layoutlmv2/feature_extraction_layoutlmv2.py src/transformers/models/layoutlmv2/modeling_layoutlmv2.py src/transformers/models/layoutlmv2/processing_layoutlmv2.py src/transformers/models/layoutlmv2/tokenization_layoutlmv2.py src/transformers/models/layoutlmv2/tokenization_layoutlmv2_fast.py src/transformers/models/led/configuration_led.py src/transformers/models/led/modeling_led.py src/transformers/models/longformer/modeling_longformer.py src/transformers/models/luke/configuration_luke.py src/transformers/models/luke/modeling_luke.py src/transformers/models/luke/tokenization_luke.py src/transformers/models/lxmert/configuration_lxmert.py src/transformers/models/m2m_100/configuration_m2m_100.py src/transformers/models/m2m_100/modeling_m2m_100.py src/transformers/models/m2m_100/tokenization_m2m_100.py src/transformers/models/marian/configuration_marian.py src/transformers/models/marian/modeling_flax_marian.py src/transformers/models/marian/modeling_marian.py src/transformers/models/marian/modeling_tf_marian.py src/transformers/models/mbart/configuration_mbart.py src/transformers/models/mbart/modeling_flax_mbart.py src/transformers/models/mbart/modeling_mbart.py src/transformers/models/mbart/tokenization_mbart.py src/transformers/models/mbart/tokenization_mbart_fast.py src/transformers/models/mbart50/tokenization_mbart50.py src/transformers/models/mbart50/tokenization_mbart50_fast.py src/transformers/models/megatron_bert/configuration_megatron_bert.py src/transformers/models/megatron_bert/convert_megatron_bert_checkpoint.py src/transformers/models/megatron_bert/modeling_megatron_bert.py src/transformers/models/megatron_gpt2/convert_megatron_gpt2_checkpoint.py src/transformers/models/openai/configuration_openai.py src/transformers/models/pegasus/__init__.py src/transformers/models/pegasus/configuration_pegasus.py src/transformers/models/pegasus/modeling_flax_pegasus.py src/transformers/models/pegasus/modeling_pegasus.py src/transformers/models/pegasus/modeling_tf_pegasus.py src/transformers/models/pegasus/tokenization_pegasus_fast.py src/transformers/models/prophetnet/configuration_prophetnet.py src/transformers/models/prophetnet/modeling_prophetnet.py src/transformers/models/rag/modeling_rag.py src/transformers/models/rag/modeling_tf_rag.py src/transformers/models/reformer/configuration_reformer.py src/transformers/models/reformer/tokenization_reformer_fast.py src/transformers/models/rembert/configuration_rembert.py src/transformers/models/rembert/modeling_rembert.py src/transformers/models/rembert/tokenization_rembert_fast.py src/transformers/models/roberta/modeling_flax_roberta.py src/transformers/models/roberta/modeling_roberta.py src/transformers/models/roberta/modeling_tf_roberta.py src/transformers/models/roformer/configuration_roformer.py src/transformers/models/roformer/modeling_roformer.py src/transformers/models/speech_encoder_decoder/__init__.py src/transformers/models/speech_encoder_decoder/configuration_speech_encoder_decoder.py src/transformers/models/speech_encoder_decoder/convert_speech_to_text_wav2vec2_seq2seq_original_to_pytorch.py src/transformers/models/speech_encoder_decoder/modeling_speech_encoder_decoder.py src/transformers/models/speech_to_text/configuration_speech_to_text.py src/transformers/models/speech_to_text/feature_extraction_speech_to_text.py src/transformers/models/speech_to_text/modeling_speech_to_text.py src/transformers/models/speech_to_text_2/__init__.py src/transformers/models/speech_to_text_2/configuration_speech_to_text_2.py src/transformers/models/speech_to_text_2/modeling_speech_to_text_2.py src/transformers/models/speech_to_text_2/processing_speech_to_text_2.py src/transformers/models/speech_to_text_2/tokenization_speech_to_text_2.py src/transformers/models/splinter/configuration_splinter.py src/transformers/models/splinter/modeling_splinter.py src/transformers/models/t5/configuration_t5.py src/transformers/models/t5/modeling_flax_t5.py src/transformers/models/t5/modeling_t5.py src/transformers/models/t5/modeling_tf_t5.py src/transformers/models/t5/tokenization_t5_fast.py src/transformers/models/tapas/__init__.py src/transformers/models/tapas/configuration_tapas.py src/transformers/models/tapas/convert_tapas_original_tf_checkpoint_to_pytorch.py src/transformers/models/tapas/modeling_tapas.py src/transformers/models/tapas/tokenization_tapas.py src/transformers/models/transfo_xl/configuration_transfo_xl.py src/transformers/models/visual_bert/modeling_visual_bert.py src/transformers/models/vit/configuration_vit.py src/transformers/models/vit/convert_dino_to_pytorch.py src/transformers/models/vit/modeling_flax_vit.py src/transformers/models/vit/modeling_vit.py src/transformers/models/wav2vec2/__init__.py src/transformers/models/wav2vec2/configuration_wav2vec2.py src/transformers/models/wav2vec2/convert_wav2vec2_original_s3prl_checkpoint_to_pytorch.py src/transformers/models/wav2vec2/feature_extraction_wav2vec2.py src/transformers/models/wav2vec2/modeling_flax_wav2vec2.py src/transformers/models/wav2vec2/modeling_wav2vec2.py src/transformers/models/wav2vec2/tokenization_wav2vec2.py src/transformers/models/xlm/configuration_xlm.py src/transformers/models/xlm_roberta/tokenization_xlm_roberta.py src/transformers/models/xlm_roberta/tokenization_xlm_roberta_fast.py src/transformers/models/xlnet/configuration_xlnet.py src/transformers/models/xlnet/tokenization_xlnet_fast.py src/transformers/onnx/convert.py src/transformers/onnx/features.py src/transformers/optimization.py src/transformers/pipelines/__init__.py src/transformers/pipelines/audio_classification.py src/transformers/pipelines/automatic_speech_recognition.py src/transformers/pipelines/base.py src/transformers/pipelines/conversational.py src/transformers/pipelines/feature_extraction.py src/transformers/pipelines/fill_mask.py src/transformers/pipelines/image_classification.py src/transformers/pipelines/object_detection.py src/transformers/pipelines/question_answering.py src/transformers/pipelines/table_question_answering.py src/transformers/pipelines/text2text_generation.py src/transformers/pipelines/text_classification.py src/transformers/pipelines/text_generation.py src/transformers/pipelines/token_classification.py src/transformers/pipelines/zero_shot_classification.py src/transformers/testing_utils.py src/transformers/tokenization_utils.py src/transformers/tokenization_utils_base.py src/transformers/tokenization_utils_fast.py src/transformers/trainer.py src/transformers/trainer_callback.py src/transformers/trainer_pt_utils.py src/transformers/trainer_seq2seq.py src/transformers/trainer_utils.py src/transformers/training_args.py src/transformers/training_args_seq2seq.py src/transformers/utils/dummy_detectron2_objects.py src/transformers/utils/dummy_flax_objects.py src/transformers/utils/dummy_pt_objects.py src/transformers/utils/dummy_tf_objects.py src/transformers/utils/dummy_tokenizers_objects.py src/transformers/utils/dummy_vision_objects.py tests/deepspeed/test_deepspeed.py tests/sagemaker/conftest.py tests/sagemaker/test_multi_node_data_parallel.py tests/test_configuration_auto.py tests/test_configuration_common.py tests/test_data_collator.py tests/test_feature_extraction_auto.py tests/test_feature_extraction_layoutlmv2.py tests/test_feature_extraction_speech_to_text.py tests/test_feature_extraction_wav2vec2.py tests/test_file_utils.py tests/test_modeling_auto.py tests/test_modeling_bart.py tests/test_modeling_beit.py tests/test_modeling_bert.py tests/test_modeling_clip.py tests/test_modeling_common.py tests/test_modeling_convbert.py tests/test_modeling_deit.py tests/test_modeling_distilbert.py tests/test_modeling_encoder_decoder.py tests/test_modeling_flaubert.py tests/test_modeling_flax_albert.py tests/test_modeling_flax_bart.py tests/test_modeling_flax_beit.py tests/test_modeling_flax_distilbert.py tests/test_modeling_flax_encoder_decoder.py tests/test_modeling_flax_gpt2.py tests/test_modeling_flax_gpt_neo.py tests/test_modeling_flax_mt5.py tests/test_modeling_flax_pegasus.py tests/test_modeling_fnet.py tests/test_modeling_gpt2.py tests/test_modeling_gpt_neo.py tests/test_modeling_gptj.py tests/test_modeling_hubert.py tests/test_modeling_layoutlmv2.py tests/test_modeling_pegasus.py tests/test_modeling_rag.py tests/test_modeling_reformer.py tests/test_modeling_speech_encoder_decoder.py tests/test_modeling_speech_to_text.py tests/test_modeling_speech_to_text_2.py tests/test_modeling_tf_auto.py tests/test_modeling_tf_deberta_v2.py tests/test_modeling_tf_hubert.py tests/test_modeling_tf_pytorch.py tests/test_modeling_tf_wav2vec2.py tests/test_modeling_wav2vec2.py tests/test_onnx_v2.py tests/test_pipelines_audio_classification.py tests/test_pipelines_automatic_speech_recognition.py tests/test_pipelines_common.py tests/test_pipelines_conversational.py tests/test_pipelines_feature_extraction.py tests/test_pipelines_fill_mask.py tests/test_pipelines_image_classification.py tests/test_pipelines_object_detection.py tests/test_pipelines_question_answering.py tests/test_pipelines_summarization.py tests/test_pipelines_table_question_answering.py tests/test_pipelines_text2text_generation.py tests/test_pipelines_text_classification.py tests/test_pipelines_text_generation.py tests/test_pipelines_token_classification.py tests/test_pipelines_translation.py tests/test_pipelines_zero_shot.py tests/test_processor_layoutlmv2.py tests/test_processor_wav2vec2.py tests/test_sequence_feature_extraction_common.py tests/test_tokenization_auto.py tests/test_tokenization_byt5.py tests/test_tokenization_canine.py tests/test_tokenization_common.py tests/test_tokenization_fnet.py tests/test_tokenization_layoutlmv2.py tests/test_tokenization_luke.py tests/test_tokenization_mbart.py tests/test_tokenization_mbart50.py tests/test_tokenization_speech_to_text_2.py tests/test_tokenization_t5.py tests/test_tokenization_tapas.py tests/test_tokenization_xlm_roberta.py tests/test_trainer.py tests/test_trainer_distributed.py tests/test_trainer_tpu.py tests/test_utils_check_copies.py utils/check_copies.py utils/check_repo.py utils/notification_service.py utils/release.py utils/tests_fetcher.py
python utils/custom_init_isort.py
python utils/style_doc.py src/transformers docs/source --max_len 119
running deps_table_update
updating src/transformers/dependency_versions_table.py
python utils/check_copies.py
python utils/check_table.py
python utils/check_dummies.py
python utils/check_repo.py
Checking all models are public.
Checking all models are properly tested.
Checking all objects are properly documented.
Checking all models are in at least one auto class.
python utils/check_inits.py
python utils/tests_fetcher.py --sanity_check and fix suggested changes.
* Run black examples tests src utils
isort examples tests src utils
Skipped 1 files
make autogenerate_code
make[1]: Entering directory '/mnt/c/Users/Admin/Desktop/Home/Projects/transformers'
running deps_table_update
updating src/transformers/dependency_versions_table.py
make[1]: Leaving directory '/mnt/c/Users/Admin/Desktop/Home/Projects/transformers'
make extra_style_checks
make[1]: Entering directory '/mnt/c/Users/Admin/Desktop/Home/Projects/transformers'
python utils/custom_init_isort.py
python utils/style_doc.py src/transformers docs/source --max_len 119
make[1]: Leaving directory '/mnt/c/Users/Admin/Desktop/Home/Projects/transformers' for reformatting code.
* Add installation dependencies for examples/research_projects/fsner.
* Add support to pass in variable numbers of examples to FSNER model.
* Retrieve start_token_id and end_token_id from tokenizer instead of hardcoding in the FSNER model.
* Run black examples tests src utils
isort examples tests src utils
Skipped 1 files
make autogenerate_code
make[1]: Entering directory '/home/saif/transformers'
running deps_table_update
updating src/transformers/dependency_versions_table.py
make[1]: Leaving directory '/home/saif/transformers'
make extra_style_checks
make[1]: Entering directory '/home/saif/transformers'
python utils/custom_init_isort.py
python utils/style_doc.py src/transformers docs/source --max_len 119
make[1]: Leaving directory '/home/saif/transformers' for FSNER
* Update FSNER readme.md with a header image.
* Update FSNER readme
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
Co-authored-by: Stefan Schweter <stefan@schweter.it>
2021-10-05 22:47:11 -04:00
Yih-Dar
a6ea244f99
Fix: save checkpoint after each epoch and push checkpoint to the hub ( #13872 )
...
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2021-10-05 16:30:13 +05:30
Nathan Raw
cc0a415e2f
✨ update image classification example ( #13824 )
...
* ✨ update image classification example
* 📌 update reqs
2021-10-04 11:49:51 -07:00
Anton Lozhkov
4213728067
[Examples] Add an official audio classification example ( #13722 )
...
* Restore broken merge
* Additional args, DDP, remove CommonLanguage
* Update examples for V100, add training results
* Style
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Remove custom datasets for simplicity, apply suggestions from code review
* Add the attention_mask flag, reorganize README
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2021-10-01 18:52:45 +02:00
Patrick von Platen
44eb8bdeea
map only on one process ( #13810 )
2021-09-30 18:52:53 +02:00
Suraj Patil
7db2a79b38
[examples/flax] use Repository API for push_to_hub ( #13672 )
...
* use Repository for push_to_hub
* update readme
* update other flax scripts
* update readme
* update qa example
* fix push_to_hub call
* fix typo
* fix more typos
* update readme
* use abosolute path to get repo name
* fix glue script
2021-09-30 16:38:07 +05:30
Stas Bekman
b90096fe14
[examples run_glue.py] missing requirements scipy, sklearn ( #13768 )
...
* missing requirement
* list both
2021-09-29 13:45:19 -07:00
Lysandre
11c69b8045
Docs for version v4.11.0
2021-09-27 14:19:38 -04:00
Lysandre
dc193c906d
Release: v4.11.0
2021-09-27 14:14:09 -04:00
Sylvain Gugger
044eff5bf0
Update requirements for speech example ( #13745 )
2021-09-26 09:02:45 +02:00
Patrick von Platen
469b80d4e7
Update README.md
2021-09-24 18:53:58 +02:00
Patrick von Platen
493643fff8
up ( #13733 )
2021-09-24 18:32:35 +02:00
Gunjan Chhablani
38580455de
Add model card creation snippet to example scripts ( #13730 )
...
* Update run_glue.py
* Update run_glue.py
* Add model creation snippet to other scripts
* Fix style
2021-09-24 15:51:46 +02:00
Patrick von Platen
95f888fd6a
Update README.md
2021-09-24 09:53:37 +02:00
Patrick von Platen
4a320f6c9a
[ASR] Add official ASR CTC example to examples/pytorch/speech-recognition ( #13620 )
...
* up
* rename
* add asr example
* add auto feature extractor
* some more fixes
* correct layerdrop
* correct for multi-gpu dist
* clean up
* refactor
* refactor
* more fixes
* more fixes
* clean-up
* finish
* up
* Apply suggestions from code review
* fix isort
* update
* up
* add note
* apply surajs suggestions
* Apply suggestions from code review
Co-authored-by: Suraj Patil <surajp815@gmail.com>
* isort
* small change
* Apply suggestions from code review
Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
* Apply suggestions from code review
Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
* add hubert
* Update examples/pytorch/speech-recognition/run_speech_recognition_ctc.py
Co-authored-by: Suraj Patil <surajp815@gmail.com>
Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
2021-09-24 07:01:11 +02:00
Md Saiful Islam Sayef
f888e5c372
Add FSNER example in research_projects ( #13712 )
...
* Add example use of few-shot named entity recognition model in research_projects folder.
* Apply suggestions from code review
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update fsner example README.md.
- Change wrong import FSNERTokenizerWrapper to FSNERTokenizerUtils in the example code
- Add a link to the model identifier
* Update examples/research_projects/fsner/src/fsner/model.py
Fix spelling mistake in the default parameter of pretrained model name.
Co-authored-by: Stefan Schweter <stefan@schweter.it>
* Add example use of few-shot named entity recognition model in research_projects folder.
* Apply suggestions from code review
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update fsner example README.md.
- Change wrong import FSNERTokenizerWrapper to FSNERTokenizerUtils in the example code
- Add a link to the model identifier
* Update examples/research_projects/fsner/src/fsner/model.py
Fix spelling mistake in the default parameter of pretrained model name.
Co-authored-by: Stefan Schweter <stefan@schweter.it>
* Run Checking/fixing examples/flax/language-modeling/run_clm_flax.py examples/flax/question-answering/run_qa.py examples/flax/question-answering/utils_qa.py examples/flax/token-classification/run_flax_ner.py examples/legacy/multiple_choice/utils_multiple_choice.py examples/legacy/seq2seq/seq2seq_trainer.py examples/legacy/token-classification/utils_ner.py examples/pytorch/image-classification/run_image_classification.py examples/pytorch/language-modeling/run_clm.py examples/pytorch/language-modeling/run_clm_no_trainer.py examples/pytorch/language-modeling/run_mlm.py examples/pytorch/language-modeling/run_mlm_no_trainer.py examples/pytorch/language-modeling/run_plm.py examples/pytorch/multiple-choice/run_swag.py examples/pytorch/multiple-choice/run_swag_no_trainer.py examples/pytorch/question-answering/run_qa.py examples/pytorch/question-answering/run_qa_beam_search.py examples/pytorch/question-answering/run_qa_beam_search_no_trainer.py examples/pytorch/question-answering/run_qa_no_trainer.py examples/pytorch/summarization/run_summarization.py examples/pytorch/summarization/run_summarization_no_trainer.py examples/pytorch/test_examples.py examples/pytorch/text-classification/run_glue.py examples/pytorch/text-classification/run_glue_no_trainer.py examples/pytorch/text-classification/run_xnli.py examples/pytorch/token-classification/run_ner.py examples/pytorch/token-classification/run_ner_no_trainer.py examples/pytorch/translation/run_translation.py examples/pytorch/translation/run_translation_no_trainer.py examples/research_projects/adversarial/utils_hans.py examples/research_projects/distillation/grouped_batch_sampler.py examples/research_projects/fsner/setup.py examples/research_projects/fsner/src/fsner/__init__.py examples/research_projects/fsner/src/fsner/model.py examples/research_projects/fsner/src/fsner/tokenizer_utils.py examples/research_projects/jax-projects/big_bird/evaluate.py examples/research_projects/jax-projects/hybrid_clip/run_hybrid_clip.py examples/tensorflow/language-modeling/run_clm.py examples/tensorflow/multiple-choice/run_swag.py examples/tensorflow/question-answering/run_qa.py examples/tensorflow/summarization/run_summarization.py examples/tensorflow/text-classification/run_glue.py examples/tensorflow/translation/run_translation.py src/transformers/__init__.py src/transformers/commands/add_new_model.py src/transformers/configuration_utils.py src/transformers/convert_slow_tokenizer.py src/transformers/data/__init__.py src/transformers/data/data_collator.py src/transformers/data/datasets/glue.py src/transformers/data/datasets/language_modeling.py src/transformers/data/datasets/squad.py src/transformers/deepspeed.py src/transformers/dependency_versions_table.py src/transformers/feature_extraction_sequence_utils.py src/transformers/file_utils.py src/transformers/generation_flax_utils.py src/transformers/generation_logits_process.py src/transformers/generation_tf_utils.py src/transformers/generation_utils.py src/transformers/integrations.py src/transformers/modelcard.py src/transformers/modeling_flax_utils.py src/transformers/modeling_outputs.py src/transformers/modeling_tf_utils.py src/transformers/modeling_utils.py src/transformers/models/__init__.py src/transformers/models/albert/__init__.py src/transformers/models/albert/modeling_albert.py src/transformers/models/albert/modeling_flax_albert.py src/transformers/models/albert/tokenization_albert_fast.py src/transformers/models/auto/__init__.py src/transformers/models/auto/auto_factory.py src/transformers/models/auto/configuration_auto.py src/transformers/models/auto/dynamic.py src/transformers/models/auto/feature_extraction_auto.py src/transformers/models/auto/modeling_auto.py src/transformers/models/auto/modeling_flax_auto.py src/transformers/models/auto/modeling_tf_auto.py src/transformers/models/auto/tokenization_auto.py src/transformers/models/bart/configuration_bart.py src/transformers/models/bart/modeling_bart.py src/transformers/models/bart/modeling_flax_bart.py src/transformers/models/bart/modeling_tf_bart.py src/transformers/models/barthez/tokenization_barthez_fast.py src/transformers/models/beit/__init__.py src/transformers/models/beit/configuration_beit.py src/transformers/models/beit/modeling_beit.py src/transformers/models/beit/modeling_flax_beit.py src/transformers/models/bert/configuration_bert.py src/transformers/models/bert/modeling_bert.py src/transformers/models/bert/modeling_flax_bert.py src/transformers/models/bert_generation/configuration_bert_generation.py src/transformers/models/bert_generation/modeling_bert_generation.py src/transformers/models/big_bird/configuration_big_bird.py src/transformers/models/big_bird/modeling_big_bird.py src/transformers/models/big_bird/modeling_flax_big_bird.py src/transformers/models/big_bird/tokenization_big_bird_fast.py src/transformers/models/bigbird_pegasus/configuration_bigbird_pegasus.py src/transformers/models/bigbird_pegasus/modeling_bigbird_pegasus.py src/transformers/models/blenderbot/configuration_blenderbot.py src/transformers/models/blenderbot/modeling_blenderbot.py src/transformers/models/blenderbot/modeling_tf_blenderbot.py src/transformers/models/blenderbot_small/configuration_blenderbot_small.py src/transformers/models/blenderbot_small/modeling_blenderbot_small.py src/transformers/models/blenderbot_small/modeling_tf_blenderbot_small.py src/transformers/models/byt5/tokenization_byt5.py src/transformers/models/camembert/tokenization_camembert_fast.py src/transformers/models/canine/configuration_canine.py src/transformers/models/canine/modeling_canine.py src/transformers/models/clip/configuration_clip.py src/transformers/models/clip/convert_clip_original_pytorch_to_hf.py src/transformers/models/clip/modeling_clip.py src/transformers/models/clip/modeling_flax_clip.py src/transformers/models/clip/tokenization_clip.py src/transformers/models/convbert/modeling_convbert.py src/transformers/models/ctrl/configuration_ctrl.py src/transformers/models/deberta/modeling_tf_deberta.py src/transformers/models/deberta_v2/__init__.py src/transformers/models/deberta_v2/modeling_deberta_v2.py src/transformers/models/deberta_v2/modeling_tf_deberta_v2.py src/transformers/models/deit/configuration_deit.py src/transformers/models/deit/modeling_deit.py src/transformers/models/detr/configuration_detr.py src/transformers/models/detr/modeling_detr.py src/transformers/models/distilbert/__init__.py src/transformers/models/distilbert/configuration_distilbert.py src/transformers/models/distilbert/modeling_distilbert.py src/transformers/models/distilbert/modeling_flax_distilbert.py src/transformers/models/dpr/configuration_dpr.py src/transformers/models/dpr/modeling_dpr.py src/transformers/models/electra/modeling_electra.py src/transformers/models/electra/modeling_flax_electra.py src/transformers/models/encoder_decoder/__init__.py src/transformers/models/encoder_decoder/modeling_encoder_decoder.py src/transformers/models/encoder_decoder/modeling_flax_encoder_decoder.py src/transformers/models/flaubert/configuration_flaubert.py src/transformers/models/flaubert/modeling_flaubert.py src/transformers/models/fnet/__init__.py src/transformers/models/fnet/configuration_fnet.py src/transformers/models/fnet/convert_fnet_original_flax_checkpoint_to_pytorch.py src/transformers/models/fnet/modeling_fnet.py src/transformers/models/fnet/tokenization_fnet.py src/transformers/models/fnet/tokenization_fnet_fast.py src/transformers/models/fsmt/configuration_fsmt.py src/transformers/models/fsmt/modeling_fsmt.py src/transformers/models/funnel/configuration_funnel.py src/transformers/models/gpt2/__init__.py src/transformers/models/gpt2/configuration_gpt2.py src/transformers/models/gpt2/modeling_flax_gpt2.py src/transformers/models/gpt2/modeling_gpt2.py src/transformers/models/gpt2/modeling_tf_gpt2.py src/transformers/models/gpt_neo/configuration_gpt_neo.py src/transformers/models/gpt_neo/modeling_gpt_neo.py src/transformers/models/gptj/__init__.py src/transformers/models/gptj/configuration_gptj.py src/transformers/models/gptj/modeling_gptj.py src/transformers/models/herbert/tokenization_herbert_fast.py src/transformers/models/hubert/__init__.py src/transformers/models/hubert/configuration_hubert.py src/transformers/models/hubert/convert_hubert_original_s3prl_checkpoint_to_pytorch.py src/transformers/models/hubert/modeling_hubert.py src/transformers/models/hubert/modeling_tf_hubert.py src/transformers/models/ibert/modeling_ibert.py src/transformers/models/layoutlm/__init__.py src/transformers/models/layoutlm/configuration_layoutlm.py src/transformers/models/layoutlm/modeling_layoutlm.py src/transformers/models/layoutlmv2/__init__.py src/transformers/models/layoutlmv2/configuration_layoutlmv2.py src/transformers/models/layoutlmv2/feature_extraction_layoutlmv2.py src/transformers/models/layoutlmv2/modeling_layoutlmv2.py src/transformers/models/layoutlmv2/processing_layoutlmv2.py src/transformers/models/layoutlmv2/tokenization_layoutlmv2.py src/transformers/models/layoutlmv2/tokenization_layoutlmv2_fast.py src/transformers/models/led/configuration_led.py src/transformers/models/led/modeling_led.py src/transformers/models/longformer/modeling_longformer.py src/transformers/models/luke/configuration_luke.py src/transformers/models/luke/modeling_luke.py src/transformers/models/luke/tokenization_luke.py src/transformers/models/lxmert/configuration_lxmert.py src/transformers/models/m2m_100/configuration_m2m_100.py src/transformers/models/m2m_100/modeling_m2m_100.py src/transformers/models/m2m_100/tokenization_m2m_100.py src/transformers/models/marian/configuration_marian.py src/transformers/models/marian/modeling_flax_marian.py src/transformers/models/marian/modeling_marian.py src/transformers/models/marian/modeling_tf_marian.py src/transformers/models/mbart/configuration_mbart.py src/transformers/models/mbart/modeling_flax_mbart.py src/transformers/models/mbart/modeling_mbart.py src/transformers/models/mbart/tokenization_mbart.py src/transformers/models/mbart/tokenization_mbart_fast.py src/transformers/models/mbart50/tokenization_mbart50.py src/transformers/models/mbart50/tokenization_mbart50_fast.py src/transformers/models/megatron_bert/configuration_megatron_bert.py src/transformers/models/megatron_bert/convert_megatron_bert_checkpoint.py src/transformers/models/megatron_bert/modeling_megatron_bert.py src/transformers/models/megatron_gpt2/convert_megatron_gpt2_checkpoint.py src/transformers/models/openai/configuration_openai.py src/transformers/models/pegasus/__init__.py src/transformers/models/pegasus/configuration_pegasus.py src/transformers/models/pegasus/modeling_flax_pegasus.py src/transformers/models/pegasus/modeling_pegasus.py src/transformers/models/pegasus/modeling_tf_pegasus.py src/transformers/models/pegasus/tokenization_pegasus_fast.py src/transformers/models/prophetnet/configuration_prophetnet.py src/transformers/models/prophetnet/modeling_prophetnet.py src/transformers/models/rag/modeling_rag.py src/transformers/models/rag/modeling_tf_rag.py src/transformers/models/reformer/configuration_reformer.py src/transformers/models/reformer/tokenization_reformer_fast.py src/transformers/models/rembert/configuration_rembert.py src/transformers/models/rembert/modeling_rembert.py src/transformers/models/rembert/tokenization_rembert_fast.py src/transformers/models/roberta/modeling_flax_roberta.py src/transformers/models/roberta/modeling_roberta.py src/transformers/models/roberta/modeling_tf_roberta.py src/transformers/models/roformer/configuration_roformer.py src/transformers/models/roformer/modeling_roformer.py src/transformers/models/speech_encoder_decoder/__init__.py src/transformers/models/speech_encoder_decoder/configuration_speech_encoder_decoder.py src/transformers/models/speech_encoder_decoder/convert_speech_to_text_wav2vec2_seq2seq_original_to_pytorch.py src/transformers/models/speech_encoder_decoder/modeling_speech_encoder_decoder.py src/transformers/models/speech_to_text/configuration_speech_to_text.py src/transformers/models/speech_to_text/feature_extraction_speech_to_text.py src/transformers/models/speech_to_text/modeling_speech_to_text.py src/transformers/models/speech_to_text_2/__init__.py src/transformers/models/speech_to_text_2/configuration_speech_to_text_2.py src/transformers/models/speech_to_text_2/modeling_speech_to_text_2.py src/transformers/models/speech_to_text_2/processing_speech_to_text_2.py src/transformers/models/speech_to_text_2/tokenization_speech_to_text_2.py src/transformers/models/splinter/configuration_splinter.py src/transformers/models/splinter/modeling_splinter.py src/transformers/models/t5/configuration_t5.py src/transformers/models/t5/modeling_flax_t5.py src/transformers/models/t5/modeling_t5.py src/transformers/models/t5/modeling_tf_t5.py src/transformers/models/t5/tokenization_t5_fast.py src/transformers/models/tapas/__init__.py src/transformers/models/tapas/configuration_tapas.py src/transformers/models/tapas/convert_tapas_original_tf_checkpoint_to_pytorch.py src/transformers/models/tapas/modeling_tapas.py src/transformers/models/tapas/tokenization_tapas.py src/transformers/models/transfo_xl/configuration_transfo_xl.py src/transformers/models/visual_bert/modeling_visual_bert.py src/transformers/models/vit/configuration_vit.py src/transformers/models/vit/convert_dino_to_pytorch.py src/transformers/models/vit/modeling_flax_vit.py src/transformers/models/vit/modeling_vit.py src/transformers/models/wav2vec2/__init__.py src/transformers/models/wav2vec2/configuration_wav2vec2.py src/transformers/models/wav2vec2/convert_wav2vec2_original_s3prl_checkpoint_to_pytorch.py src/transformers/models/wav2vec2/feature_extraction_wav2vec2.py src/transformers/models/wav2vec2/modeling_flax_wav2vec2.py src/transformers/models/wav2vec2/modeling_wav2vec2.py src/transformers/models/wav2vec2/tokenization_wav2vec2.py src/transformers/models/xlm/configuration_xlm.py src/transformers/models/xlm_roberta/tokenization_xlm_roberta.py src/transformers/models/xlm_roberta/tokenization_xlm_roberta_fast.py src/transformers/models/xlnet/configuration_xlnet.py src/transformers/models/xlnet/tokenization_xlnet_fast.py src/transformers/onnx/convert.py src/transformers/onnx/features.py src/transformers/optimization.py src/transformers/pipelines/__init__.py src/transformers/pipelines/audio_classification.py src/transformers/pipelines/automatic_speech_recognition.py src/transformers/pipelines/base.py src/transformers/pipelines/conversational.py src/transformers/pipelines/feature_extraction.py src/transformers/pipelines/fill_mask.py src/transformers/pipelines/image_classification.py src/transformers/pipelines/object_detection.py src/transformers/pipelines/question_answering.py src/transformers/pipelines/table_question_answering.py src/transformers/pipelines/text2text_generation.py src/transformers/pipelines/text_classification.py src/transformers/pipelines/text_generation.py src/transformers/pipelines/token_classification.py src/transformers/pipelines/zero_shot_classification.py src/transformers/testing_utils.py src/transformers/tokenization_utils.py src/transformers/tokenization_utils_base.py src/transformers/tokenization_utils_fast.py src/transformers/trainer.py src/transformers/trainer_callback.py src/transformers/trainer_pt_utils.py src/transformers/trainer_seq2seq.py src/transformers/trainer_utils.py src/transformers/training_args.py src/transformers/training_args_seq2seq.py src/transformers/utils/dummy_detectron2_objects.py src/transformers/utils/dummy_flax_objects.py src/transformers/utils/dummy_pt_objects.py src/transformers/utils/dummy_tf_objects.py src/transformers/utils/dummy_tokenizers_objects.py src/transformers/utils/dummy_vision_objects.py tests/deepspeed/test_deepspeed.py tests/sagemaker/conftest.py tests/sagemaker/test_multi_node_data_parallel.py tests/test_configuration_auto.py tests/test_configuration_common.py tests/test_data_collator.py tests/test_feature_extraction_auto.py tests/test_feature_extraction_layoutlmv2.py tests/test_feature_extraction_speech_to_text.py tests/test_feature_extraction_wav2vec2.py tests/test_file_utils.py tests/test_modeling_auto.py tests/test_modeling_bart.py tests/test_modeling_beit.py tests/test_modeling_bert.py tests/test_modeling_clip.py tests/test_modeling_common.py tests/test_modeling_convbert.py tests/test_modeling_deit.py tests/test_modeling_distilbert.py tests/test_modeling_encoder_decoder.py tests/test_modeling_flaubert.py tests/test_modeling_flax_albert.py tests/test_modeling_flax_bart.py tests/test_modeling_flax_beit.py tests/test_modeling_flax_distilbert.py tests/test_modeling_flax_encoder_decoder.py tests/test_modeling_flax_gpt2.py tests/test_modeling_flax_gpt_neo.py tests/test_modeling_flax_mt5.py tests/test_modeling_flax_pegasus.py tests/test_modeling_fnet.py tests/test_modeling_gpt2.py tests/test_modeling_gpt_neo.py tests/test_modeling_gptj.py tests/test_modeling_hubert.py tests/test_modeling_layoutlmv2.py tests/test_modeling_pegasus.py tests/test_modeling_rag.py tests/test_modeling_reformer.py tests/test_modeling_speech_encoder_decoder.py tests/test_modeling_speech_to_text.py tests/test_modeling_speech_to_text_2.py tests/test_modeling_tf_auto.py tests/test_modeling_tf_deberta_v2.py tests/test_modeling_tf_hubert.py tests/test_modeling_tf_pytorch.py tests/test_modeling_tf_wav2vec2.py tests/test_modeling_wav2vec2.py tests/test_onnx_v2.py tests/test_pipelines_audio_classification.py tests/test_pipelines_automatic_speech_recognition.py tests/test_pipelines_common.py tests/test_pipelines_conversational.py tests/test_pipelines_feature_extraction.py tests/test_pipelines_fill_mask.py tests/test_pipelines_image_classification.py tests/test_pipelines_object_detection.py tests/test_pipelines_question_answering.py tests/test_pipelines_summarization.py tests/test_pipelines_table_question_answering.py tests/test_pipelines_text2text_generation.py tests/test_pipelines_text_classification.py tests/test_pipelines_text_generation.py tests/test_pipelines_token_classification.py tests/test_pipelines_translation.py tests/test_pipelines_zero_shot.py tests/test_processor_layoutlmv2.py tests/test_processor_wav2vec2.py tests/test_sequence_feature_extraction_common.py tests/test_tokenization_auto.py tests/test_tokenization_byt5.py tests/test_tokenization_canine.py tests/test_tokenization_common.py tests/test_tokenization_fnet.py tests/test_tokenization_layoutlmv2.py tests/test_tokenization_luke.py tests/test_tokenization_mbart.py tests/test_tokenization_mbart50.py tests/test_tokenization_speech_to_text_2.py tests/test_tokenization_t5.py tests/test_tokenization_tapas.py tests/test_tokenization_xlm_roberta.py tests/test_trainer.py tests/test_trainer_distributed.py tests/test_trainer_tpu.py tests/test_utils_check_copies.py utils/check_copies.py utils/check_repo.py utils/notification_service.py utils/release.py utils/tests_fetcher.py
python utils/custom_init_isort.py
python utils/style_doc.py src/transformers docs/source --max_len 119
running deps_table_update
updating src/transformers/dependency_versions_table.py
python utils/check_copies.py
python utils/check_table.py
python utils/check_dummies.py
python utils/check_repo.py
Checking all models are public.
Checking all models are properly tested.
Checking all objects are properly documented.
Checking all models are in at least one auto class.
python utils/check_inits.py
python utils/tests_fetcher.py --sanity_check and fix suggested changes.
* Run black examples tests src utils
isort examples tests src utils
Skipped 1 files
make autogenerate_code
make[1]: Entering directory '/mnt/c/Users/Admin/Desktop/Home/Projects/transformers'
running deps_table_update
updating src/transformers/dependency_versions_table.py
make[1]: Leaving directory '/mnt/c/Users/Admin/Desktop/Home/Projects/transformers'
make extra_style_checks
make[1]: Entering directory '/mnt/c/Users/Admin/Desktop/Home/Projects/transformers'
python utils/custom_init_isort.py
python utils/style_doc.py src/transformers docs/source --max_len 119
make[1]: Leaving directory '/mnt/c/Users/Admin/Desktop/Home/Projects/transformers' for reformatting code.
* Add installation dependencies for examples/research_projects/fsner.
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
Co-authored-by: Stefan Schweter <stefan@schweter.it>
2021-09-23 17:04:15 -04:00
Sylvain Gugger
27d4639779
Make gradient_checkpointing a training argument ( #13657 )
...
* Make gradient_checkpointing a training argument
* Update src/transformers/modeling_utils.py
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
* Update src/transformers/configuration_utils.py
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
* Fix tests
* Style
* document Gradient Checkpointing as a performance feature
* Small rename
* PoC for not using the config
* Adapt BC to new PoC
* Forgot to save
* Rollout changes to all other models
* Fix typo
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
Co-authored-by: Stas Bekman <stas@stason.org>
2021-09-22 07:51:38 -04:00
Sylvain Gugger
b7d264be0d
Add push_to_hub to no_trainer examples ( #13659 )
...
* Add push_to_hub to no_trainer examples
* Quality
* Document integration
* Roll out to other examples
2021-09-21 13:13:30 -04:00
Kamal Raj
78807d86eb
[FLAX] Question Answering Example ( #13649 )
...
* flax qa example
* Updated README: Added Large model
* added utils_qa.py FULL_COPIES
* Updates:
1. Copyright Year updated
2. added dtype arg
3. passing seed and dtype to load model
4. Check eval flag before running eval
* updated README
* updated code comment
2021-09-21 18:34:48 +05:30
Lowin
28d5700aae
fix research_projects/mlm_wwm readme.md examples ( #13646 )
...
the variables of run example is not correct
2021-09-20 15:01:35 -04:00
Suraj Patil
87d5057d86
fix typo ( #13647 )
2021-09-20 13:22:26 +05:30
Patrick von Platen
95f933ea85
[Pretrained Model] Add resize_position_embeddings ( #13559 )
...
* finish
* delete bogus file
* correct some stuff
* finish
* finish
2021-09-15 19:03:56 +02:00
elishowk
c783e14887
upgrade sentencepiece version ( #13564 )
2021-09-15 15:25:03 +02:00
Avital Oliver
51e5eca612
Add long overdue link to the Google TRC project ( #13501 )
...
* Add long-overdue link to the Google TRC project
* Apply suggestions from code review
Co-authored-by: Suraj Patil <surajp815@gmail.com>
Co-authored-by: Stefan Schweter <stefan@schweter.it>
2021-09-14 13:41:55 +05:30
Stefan Schweter
09549aa18c
examples: minor fixes in flax example readme ( #13502 )
2021-09-10 11:45:57 +05:30
Aleksander Smywiński-Pohl
008c2d0b7a
Fix typo in documentation ( #13494 )
...
* Fix typo in deepspeed documentation
* Add missing import in deepspeed configuration
* Fix path in translation examples
2021-09-09 08:00:05 -04:00
Kamal Raj
1c191efc3a
flax ner example ( #13365 )
...
* flax ner example
* added task to README
* updated readme
* 1. ArgumentParser -> HfArgumentParser
2. step-wise logging,eval and save
* added requirements.txt
* added progress bar
* updated README
* added check_min_version
* updated training data permuattion with JAX
* added metric lib to requirements
* updated readme table
* fixed imports
2021-09-09 10:12:57 +05:30
Nathan Raw
79815090ea
Fix img classification tests ( #13456 )
...
* ✅ Update image-classification example's tests
* 🔥 remove cats_and_dogs test samples
* 💄 fix flake8
2021-09-07 05:58:45 -04:00
Suraj Patil
2dd975b235
skip image classification test ( #13451 )
2021-09-06 21:46:25 +05:30
Suraj Patil
6b29bff852
add torchvision in example test requirements ( #13438 )
2021-09-06 15:17:54 +02:00
Nathan Raw
76c4d8bf26
✨ Add PyTorch image classification example ( #13134 )
...
* ✨ add pytorch image classification example
* 🔥 remove utils.py
* 💄 fix flake8 style issues
* 🔥 remove unnecessary line
* ✨ limit dataset sizes
* 📌 update reqs
* 🎨 restructure - use datasets lib
* 🎨 import transforms directly
* 📝 add comments
* 💄 style
* 🔥 remove flag
* 📌 update requirement warning
* 📝 add vision README.md
* 📝 update README.md
* 📝 update README.md
* 🎨 add image-classification tag to model card
* 🚚 rename vision ➡️ image-classification
* 📝 update image-classification README.md
2021-09-02 13:29:42 -06:00
Eduardo Gonzalez Ponferrada
0a22335e66
[Flax/run_hybrid_clip] Fix duplicating images when captions_per_image exceeds the number of captions, enable truncation
2021-09-02 11:19:49 +05:30
Lysandre
5ee67a4412
Docs for v4.10.0
2021-08-31 16:02:31 +02:00
Lysandre
d12bbe4942
Release: v4.10.0
2021-08-31 15:53:10 +02:00
Sylvain Gugger
c76de1053e
Add generate kwargs to Seq2SeqTrainingArguments ( #13339 )
...
* Add generate kwargs to Seq2SeqTrainingArguments
* typo
* Address review comments + doc
* Style
2021-08-31 08:42:00 -04:00
Matt
702f4a49cd
Fixed CLM model still using MODEL_FOR_MASKED_LM_MAPPING ( #13002 )
2021-08-31 13:21:39 +01:00
Sylvain Gugger
139e830158
Update label2id in the model config for run_glue ( #13334 )
2021-08-30 10:35:09 -04:00
Stefan Schweter
4046e66e40
examples: only use keep_linebreaks when reading TXT files ( #13320 )
...
* examples: only use keep_linebreaks when reading TXT files for all CLM examples
* examples: only use keep_linebreaks when reading TXT files for all CLM examples
* examples: only use keep_linebreaks when reading TXT files for all CLM examples
2021-08-28 16:22:29 +02:00
Patrick von Platen
2bef3433e5
[Flax] Correct all return tensors to numpy ( #13307 )
...
* fix_torch_device_generate_test
* remove @
* finish find and replace
2021-08-27 17:38:34 +02:00
Stefan Schweter
319d840b46
examples: add keep_linebreaks option to CLM examples ( #13150 )
...
* examples: add keep_linebreaks option to text dataset loader for all CLM examples
* examples: introduce new keep_linebreaks option as data argument in CLM examples
2021-08-27 11:35:45 +02:00
dependabot[bot]
0245cee469
Bump notebook from 6.1.5 to 6.4.1 in /examples/research_projects/lxmert ( #13226 )
...
Bumps [notebook](http://jupyter.org ) from 6.1.5 to 6.4.1.
---
updated-dependencies:
- dependency-name: notebook
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-08-24 09:52:39 -04:00
Allan Lin
91ff480e26
Update namespaces inside torch.utils.data to the latest. ( #13167 )
...
* Update torch.utils.data namespaces to the latest.
* Format
* Update Dataloader.
* Style
2021-08-19 14:29:51 +02:00
Suraj Patil
f5cd27694a
[FlaxCLIP] allow passing params to image and text feature methods ( #13099 )
...
* allow passing params to image and text feature method
* ifx for hybrid clip as well
2021-08-12 18:35:01 +05:30
Sylvain Gugger
9a498c37a2
Rely on huggingface_hub for common tools ( #13100 )
...
* Remove hf_api module and use hugginface_hub
* Style
* Fix to test_fetcher
* Quality
2021-08-12 14:59:02 +02:00
Gunjan Chhablani
c71f73f438
Add VisualBERT demo notebook ( #12263 )
...
* Initialize VisualBERT demo
* Update demo
* Add commented URL
* Update README
* Update README
2021-08-11 10:10:59 -04:00
Patrick von Platen
13a9c9a354
[Flax] Refactor gpt2 & bert example docs ( #13024 )
...
* fix_torch_device_generate_test
* remove @
* improve docs for clm
* speed-ups
* correct t5 example as well
* push final touches
* Update examples/flax/language-modeling/README.md
* correct docs for mlm
* Update examples/flax/language-modeling/README.md
Co-authored-by: Patrick von Platen <patrick@huggingface.co>
2021-08-09 13:37:50 +02:00
abhishek thakur
3ff2cde5ca
tfhub.de -> tfhub.dev ( #12565 )
2021-08-09 08:11:17 +02:00
Patrick von Platen
24cbf6bc5a
Update README.md
2021-08-08 17:11:19 +02:00
Sylvain Gugger
7fcee113c1
Tpu tie weights ( #13030 )
...
* Fix tied weights on TPU
* Manually tie weights in no trainer examples
* Fix for test
* One last missing
* Gettning owned by my scripts
* Address review comments
* Fix test
* Fix tests
* Fix reformer tests
2021-08-06 20:41:39 +02:00
Patrick von Platen
2e4082364e
[Flax T5] Speed up t5 training ( #13012 )
...
* fix_torch_device_generate_test
* remove @
* update
* up
* fix
* remove f-stings
* correct readme
* up
Co-authored-by: Patrick von Platen <patrick@huggingface.co>
2021-08-06 11:21:37 +02:00
Patrick von Platen
da9754a3a0
[Flax] Align jax flax device name ( #12987 )
...
* [Flax] Align device name in docs
* make style
* fix import error
2021-08-04 16:00:09 +02:00
Chungman Lee
75b8990d90
fix typo in example/text-classification README ( #12974 )
...
* fix typo in example/text-classification README
* add space to align the table
2021-08-02 12:58:43 +02:00
Stefan Schweter
3d4b3bc3fd
examples: use correct way to get vocab size in flax lm readme ( #12947 )
2021-07-30 21:57:53 +05:30
21jun
5c673efad7
fix typo in gradient_checkpointing arg ( #12855 )
...
help for `ModelArguments.gradient_checkpointing` should be
"If True, use gradient checkpointing to save memory
at the expense of slower backward pass."
not "Whether to freeze the feature extractor layers of the model."
(which is duplicated from `freeze_feature_extractor` arg)
2021-07-30 15:06:33 +08:00
chutaklee
c164064eef
Fix distiller.py ( #12910 )
...
* fix distiller
* fix style
2021-07-29 02:11:38 +08:00
Sylvain Gugger
3ec851dc5e
Fix QA examples for roberta tokenizer ( #12928 )
2021-07-28 09:47:49 -04:00
Sylvain Gugger
fd85734e0e
Add option to set max_len in run_ner ( #12929 )
2021-07-28 09:38:12 -04:00
Elysium1436
f3d0866ed9
Correct validation_split_percentage argument from int (ex:5) to float (0.05) ( #12897 )
...
* Fixed train_test_split test_size argument
* `Seq2SeqTrainer` set max_length and num_beams only when non None (#12899 )
* set max_length and num_beams only when non None
* fix instance variables
* fix code style
* [FLAX] Minor fixes in CLM example (#12914 )
* readme: fix retrieval of vocab size for flax clm example
* examples: fix flax clm example when using training/evaluation files
* Fix module path for symbolic_trace example
Co-authored-by: cchen-dialpad <47165889+cchen-dialpad@users.noreply.github.com>
Co-authored-by: Stefan Schweter <stefan@schweter.it>
Co-authored-by: Sylvain Gugger <sylvain.gugger@gmail.com>
2021-07-27 21:01:40 -04:00
Stefan Schweter
d3c3e722d6
[FLAX] Minor fixes in CLM example ( #12914 )
...
* readme: fix retrieval of vocab size for flax clm example
* examples: fix flax clm example when using training/evaluation files
2021-07-27 19:48:04 +05:30
Matt
569f61a760
Add TF multiple choice example ( #12865 )
...
* Add new multiple-choice example, remove old one
2021-07-26 15:15:51 +01:00
Sylvain Gugger
303989de0e
Add accelerate to examples requirements ( #12888 )
2021-07-26 09:57:34 -04:00
Stas Bekman
98364ea74f
[tests] fix logging_steps requirements ( #12860 )
2021-07-23 08:05:48 -07:00
Lysandre
40de2d5a4f
Docs for v4.10.0dev0
2021-07-22 12:52:25 +02:00
Lysandre
72aee83ced
Release: v4.9.0
2021-07-22 12:11:55 +02:00
Maxwell Forbes
fcf83011df
Fix type of max_seq_length arg in run_swag.py ( #12832 )
2021-07-22 02:14:14 -04:00
Patrick von Platen
acdd78db08
Update README.md
2021-07-20 16:48:37 +02:00
Patrick von Platen
31d06729f4
Update README.md
2021-07-20 14:19:37 +02:00
Patrick von Platen
13fefdf340
Update README.md
...
cc @patil-suraj
2021-07-20 13:51:15 +02:00
fgaim
66197adc98
Flax MLM: Allow validation split when loading dataset from local file ( #12689 )
...
* Allow validation split when loading dataset from local file
* Flax clm & t5, enable validation split for datasets loaded from local file
2021-07-20 13:38:25 +02:00
Patrick von Platen
c6b9095cb2
Update README.md
2021-07-17 19:22:26 +02:00
Patrick von Platen
b4b562d834
[Wav2Vec2] Padded vectors should not allowed to be sampled ( #12764 )
...
* fix_torch_device_generate_test
* remove @
* finish
* correct script
* correct script
2021-07-16 19:07:08 +02:00
Suraj Patil
8ef3f36561
fix typos ( #12757 )
2021-07-16 16:44:59 +05:30
Patrick von Platen
a76dd7ee82
Update README.md
2021-07-16 00:16:30 +01:00
Patrick von Platen
2e9fb13fb1
[Wav2Vec2] Correctly pad mask indices for PreTraining ( #12748 )
...
* fix_torch_device_generate_test
* remove @
* start adding tests
* correct wav2vec2 pretraining
* up
* up
Co-authored-by: Patrick von Platen <patrick@huggingface.co>
2021-07-15 21:40:25 +01:00
Suraj Patil
44f5b260fe
flax model parallel training ( #12590 )
...
* update scripts
* add copyright
* add logging
* cleanup
* add z loss
* add readme
* shard description
* update readme
2021-07-14 22:55:44 +05:30
Matt
f9ac677eba
Update TF examples README ( #12703 )
...
* Update Transformers README, rename token_classification example to token-classification to be consistent with the others
* Update examples/tensorflow/README.md
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Add README for TF token classification
* Update examples/tensorflow/token-classification/README.md
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update examples/tensorflow/token-classification/README.md
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2021-07-14 15:15:25 +01:00
Patrick von Platen
f4399ec570
Update README.md
2021-07-14 12:54:31 +01:00
Matt
65bf05cd18
Adding TF translation example ( #12667 )
...
* Adding TF translation example
* Fixes and style pass for TF translation example
* Remove unused postprocess_text copied from run_summarization
* Adding README
* Review fixes
* Move changes to model.config to after we've initialized the model
2021-07-13 19:08:25 +01:00
Nick Doiron
5803a2a7ac
Add ByT5 option to example run_t5_mlm_flax.py ( #12634 )
...
* Allow ByT5 type in Flax T5 script
* use T5TokenizerFast
* change up tokenizer config
* model_args
* reorder imports
* Update run_t5_mlm_flax.py
2021-07-13 13:39:57 +01:00
Omar Sanseviero
c523b241c2
Update timeline for Flax event evaluation
2021-07-12 21:24:58 +02:00
Matt
379f649434
TF summarization example ( #12617 )
...
* Adding a TF summarization example
* Style pass
* Style fixes
* Updates for review comments
* Adding README
* Style pass
* Remove unused import
2021-07-12 15:58:38 +01:00
