mirror of
https://github.com/huggingface/transformers.git
synced 2025-07-28 16:52:24 +06:00
1c191efc3a
* flax ner example * added task to README * updated readme * 1. ArgumentParser -> HfArgumentParser 2. step-wise logging,eval and save * added requirements.txt * added progress bar * updated README * added check_min_version * updated training data permuattion with JAX * added metric lib to requirements * updated readme table * fixed imports
74 lines
2.7 KiB
Markdown
74 lines
2.7 KiB
Markdown
<!---
|
|
Copyright 2021 The Google Flax Team Authors and HuggingFace Team. All rights reserved.
|
|
|
|
Licensed under the Apache License, Version 2.0 (the "License");
|
|
you may not use this file except in compliance with the License.
|
|
You may obtain a copy of the License at
|
|
|
|
http://www.apache.org/licenses/LICENSE-2.0
|
|
|
|
Unless required by applicable law or agreed to in writing, software
|
|
distributed under the License is distributed on an "AS IS" BASIS,
|
|
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
See the License for the specific language governing permissions and
|
|
limitations under the License.
|
|
-->
|
|
|
|
# Token classification examples
|
|
|
|
Fine-tuning the library models for token classification task such as Named Entity Recognition (NER), Parts-of-speech tagging (POS) or phrase extraction (CHUNKS). The main script run_flax_ner.py leverages the 🤗 Datasets library. You can easily customize it to your needs if you need extra processing on your datasets.
|
|
|
|
It will either run on a datasets hosted on our hub or with your own text files for training and validation, you might just need to add some tweaks in the data preprocessing.
|
|
|
|
The following example fine-tunes BERT on CoNLL-2003:
|
|
|
|
To begin with it is recommended to create a model repository to save the trained model and logs.
|
|
Here we call the model `"bert-ner-conll2003-test"`, but you can change the model name as you like.
|
|
|
|
You can do this either directly on [huggingface.co](https://huggingface.co/new) (assuming that
|
|
you are logged in) or via the command line:
|
|
|
|
```
|
|
huggingface-cli repo create bert-ner-conll2003-test
|
|
```
|
|
|
|
Next we clone the model repository to add the tokenizer and model files.
|
|
|
|
```
|
|
git clone https://huggingface.co/<your-username>/bert-ner-conll2003-test
|
|
```
|
|
|
|
Great, we have set up our model repository. During training, we will automatically
|
|
push the training logs and model weights to the repo.
|
|
|
|
Next, let's add a symbolic link to the `run_flax_ner.py`.
|
|
|
|
```bash
|
|
export MODEL_DIR="./bert-ner-conll2003-test"
|
|
ln -s ~/transformers/examples/flax/token-classification/run_flax_ner.py run_flax_ner.py
|
|
```
|
|
|
|
```bash
|
|
python run_flax_ner.py \
|
|
--model_name_or_path bert-base-cased \
|
|
--dataset_name conll2003 \
|
|
--max_seq_length 128 \
|
|
--learning_rate 2e-5 \
|
|
--num_train_epochs 3 \
|
|
--per_device_train_batch_size 4 \
|
|
--output_dir ${MODEL_DIR} \
|
|
--eval_steps 300 \
|
|
--push_to_hub
|
|
```
|
|
|
|
Using the command above, the script will train for 3 epochs and run eval after each epoch.
|
|
Metrics and hyperparameters are stored in Tensorflow event files in `--output_dir`.
|
|
You can see the results by running `tensorboard` in that directory:
|
|
|
|
```bash
|
|
$ tensorboard --logdir .
|
|
```
|
|
|
|
or directly on the hub under *Training metrics*.
|
|
|
|
sample Metrics - [tfhub.dev](https://tensorboard.dev/experiment/u52qsBIpQSKEEXEJd2LVYA) |