8f137b2427
* Add implementation for DataCollatorForMultipleChoice based on docs. * Add DataCollatorForMultipleChoice to import structure. * Remove custom DataCollatorForMultipleChoice implementations from example scripts. * Remove custom implementations of DataCollatorForMultipleChoice from docs in English, Spanish, Japanese and Korean. * Refactor torch version of DataCollatorForMultipleChoice to be more easily understandable. * Apply suggested changes and run make fixup. * fix copies, style and fixup * add missing documentation * nits * fix docstring * style * nits * isort --------- Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com> Co-authored-by: Arthur Zucker <arthur.zucker@gmail.com> |
||
|---|---|---|
| .. | ||
| contrastive-image-text | ||
| image-classification | ||
| language-modeling | ||
| language-modeling-tpu | ||
| multiple-choice | ||
| question-answering | ||
| summarization | ||
| text-classification | ||
| token-classification | ||
| translation | ||
| _tests_requirements.txt | ||
| README.md | ||
| test_tensorflow_examples.py | ||
Examples
This folder contains actively maintained examples of the use of 🤗 Transformers organized into different ML tasks. All examples in this folder are TensorFlow examples and are written using native Keras. If you've previously only used 🤗 Transformers via TFTrainer, we highly recommend taking a look at the new style - we think it's a big improvement!
In addition, all scripts here now support the 🤗 Datasets library - you can grab entire datasets just by changing one command-line argument!
A note on code folding
Most of these examples have been formatted with #region blocks. In IDEs such as PyCharm and VSCode, these blocks mark named regions of code that can be folded for easier viewing. If you find any of these scripts overwhelming or difficult to follow, we highly recommend beginning with all regions folded and then examining regions one at a time!
The Big Table of Tasks
Here is the list of all our examples:
| Task | Example datasets |
|---|---|
language-modeling |
WikiText-2 |
multiple-choice |
SWAG |
question-answering |
SQuAD |
summarization |
XSum |
text-classification |
GLUE |
token-classification |
CoNLL NER |
translation |
WMT |
Coming soon
- Colab notebooks to easily run through these scripts!