mirror of
https://github.com/huggingface/transformers.git
synced 2025-07-24 06:48:58 +06:00
35 lines
1023 B
Markdown
35 lines
1023 B
Markdown
# albert-chinese-large-qa
|
|
Albert large QA model pretrained from baidu webqa and baidu dureader datasets.
|
|
|
|
## Data source
|
|
+ baidu webqa 1.0
|
|
+ baidu dureader
|
|
|
|
## Traing Method
|
|
We combined the two datasets together and created a new dataset in squad format, including 705139 samples for training and 69638 samples for validation.
|
|
We finetune the model based on the albert chinese large model.
|
|
|
|
## Hyperparams
|
|
+ learning_rate 1e-5
|
|
+ max_seq_length 512
|
|
+ max_query_length 50
|
|
+ max_answer_length 300
|
|
+ doc_stride 256
|
|
+ num_train_epochs 2
|
|
+ warmup_steps 1000
|
|
+ per_gpu_train_batch_size 8
|
|
+ gradient_accumulation_steps 3
|
|
+ n_gpu 2 (Nvidia Tesla P100)
|
|
|
|
## Usage
|
|
```
|
|
from transformers import AutoModelForQuestionAnswering, BertTokenizer
|
|
|
|
model = AutoModelForQuestionAnswering.from_pretrained('wptoux/albert-chinese-large-qa')
|
|
tokenizer = BertTokenizer.from_pretrained('wptoux/albert-chinese-large-qa')
|
|
```
|
|
***Important: use BertTokenizer***
|
|
|
|
## MoreInfo
|
|
Please visit https://github.com/wptoux/albert-chinese-large-webqa for details.
|