riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-24 23:08:57 +06:00
Code Issues Actions 35 Packages Projects Releases Wiki Activity
6eb51450fa
transformers/examples
History
Matt 6eb51450fa
TF Examples Rewrite (#18451) 
* Finished QA example

* Dodge a merge conflict

* Update text classification and LM examples

* Update NER example

* New Keras metrics WIP, fix NER example

* Update NER example

* Update MC, summarization and translation examples

* Add XLA warnings when shapes are variable

* Make sure batch_size is consistently scaled by num_replicas

* Add PushToHubCallback to all models

* Add docs links for KerasMetricCallback

* Add docs links for prepare_tf_dataset and jit_compile

* Correct inferred model names

* Don't assume the dataset has 'lang'

* Don't assume the dataset has 'lang'

* Write metrics in text classification

* Add 'framework' to TrainingArguments and TFTrainingArguments

* Export metrics in all examples and add tests

* Fix training args for Flax

* Update command line args for translation test

* make fixup

* Fix accidentally running other tests in fp16

* Remove do_train/do_eval from run_clm.py

* Remove do_train/do_eval from run_mlm.py

* Add tensorflow tests to circleci

* Fix circleci

* Update examples/tensorflow/language-modeling/run_mlm.py

Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>

* Update examples/tensorflow/test_tensorflow_examples.py

Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>

* Update examples/tensorflow/translation/run_translation.py

Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>

* Update examples/tensorflow/token-classification/run_ner.py

Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>

* Fix save path for tests

* Fix some model card kwargs

* Explain the magical -1000

* Actually enable tests this time

* Skip text classification PR until we fix shape inference

* make fixup

Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
2022-08-10 16:49:51 +01:00
..
flax transformers-cli login => huggingface-cli login (#18490) 2022-08-06 09:42:55 +02:00
legacy Just re-reading the whole doc every couple of months 😬 (#18489) 2022-08-06 09:38:55 +02:00
pytorch Update no_trainer.py scripts to include accelerate gradient accumulation wrapper (#18473) 2022-08-08 15:52:47 -04:00
research_projects transformers-cli login => huggingface-cli login (#18490) 2022-08-06 09:42:55 +02:00
tensorflow TF Examples Rewrite (#18451) 2022-08-10 16:49:51 +01:00
README.md Fix ROUGE add example check and update README (#18398) 2022-08-01 11:14:49 -04:00

README.md

Examples

We host a wide range of example scripts for multiple learning frameworks. Simply choose your favorite: TensorFlow, PyTorch or JAX/Flax.

We also have some research projects, as well as some legacy examples. Note that unlike the main examples these are not actively maintained, and may require specific older versions of dependencies in order to run.

While we strive to present as many use cases as possible, the example scripts are just that - examples. It is expected that they won't work out-of-the box on your specific problem and that you will be required to change a few lines of code to adapt them to your needs. To help you with that, most of the examples fully expose the preprocessing of the data, allowing you to tweak and edit them as required.

Please discuss on the forum or in an issue a feature you would like to implement in an example before submitting a PR; we welcome bug fixes, but since we want to keep the examples as simple as possible it's unlikely that we will merge a pull request adding more functionality at the cost of readability.

Important note

Important

To make sure you can successfully run the latest versions of the example scripts, you have to install the library from source and install some example-specific requirements. To do this, execute the following steps in a new virtual environment:

git clone https://github.com/huggingface/transformers
cd transformers
pip install .

Then cd in the example folder of your choice and run

pip install -r requirements.txt

To browse the examples corresponding to released versions of 🤗 Transformers, click on the line below and then on your desired version of the library:

Examples for older versions of 🤗 Transformers

Alternatively, you can switch your cloned 🤗 Transformers to a specific version (for instance with v3.5.1) with

git checkout tags/v3.5.1

and run the example command as usual afterward.