mirror of https://github.com/huggingface/transformers.git synced 2025-07-13 09:40:06 +06:00

History

amyeroberts 57c965a8f1 Remove deprecated logic and warnings (#30743 ) * Remove deprecated logic and warnings * Add back some code that seems to be important... * Let's just add all he nllb stuff back; removing it is a bit more involved * Remove kwargs * Remove more kwargs		2024-05-17 12:15:59 +01:00
..
README.md	TF summarization example (#12617 )	2021-07-12 15:58:38 +01:00
requirements.txt	Migrate metric to Evaluate library for tensorflow examples (#18327 )	2022-07-28 14:24:27 -04:00
run_summarization.py	Remove deprecated logic and warnings (#30743 )	2024-05-17 12:15:59 +01:00

README.md

Summarization example

This script shows an example of training a summarization model with the 🤗 Transformers library. For straightforward use-cases you may be able to use these scripts without modification, although we have also included comments in the code to indicate areas that you may need to adapt to your own projects.

Multi-GPU and TPU usage

By default, these scripts use a MirroredStrategy and will use multiple GPUs effectively if they are available. TPUs can also be used by passing the name of the TPU resource with the --tpu argument.

Example command

python run_summarization.py  \
--model_name_or_path facebook/bart-base \
--dataset_name cnn_dailymail \
--dataset_config "3.0.0" \
--output_dir /tmp/tst-summarization  \
--per_device_train_batch_size 8 \
--per_device_eval_batch_size 16 \
--num_train_epochs 3 \
--do_train \
--do_eval