eb849f6604
* Rename index.mdx to index.md * With saved modifs * Address review comment * Treat all files * .mdx -> .md * Remove special char * Update utils/tests_fetcher.py Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr> --------- Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
3.9 KiB
ERNIE
Overview
ERNIE is a series of powerful models proposed by baidu, especially in Chinese tasks, including ERNIE1.0, ERNIE2.0, ERNIE3.0, ERNIE-Gram, ERNIE-health, etc.
These models are contributed by nghuyong and the official code can be found in PaddleNLP (in PaddlePaddle).
How to use
Take ernie-1.0-base-zh as an example:
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("nghuyong/ernie-1.0-base-zh")
model = AutoModel.from_pretrained("nghuyong/ernie-1.0-base-zh")
Supported Models
| Model Name | Language | Description |
|---|---|---|
| ernie-1.0-base-zh | Chinese | Layer:12, Heads:12, Hidden:768 |
| ernie-2.0-base-en | English | Layer:12, Heads:12, Hidden:768 |
| ernie-2.0-large-en | English | Layer:24, Heads:16, Hidden:1024 |
| ernie-3.0-base-zh | Chinese | Layer:12, Heads:12, Hidden:768 |
| ernie-3.0-medium-zh | Chinese | Layer:6, Heads:12, Hidden:768 |
| ernie-3.0-mini-zh | Chinese | Layer:6, Heads:12, Hidden:384 |
| ernie-3.0-micro-zh | Chinese | Layer:4, Heads:12, Hidden:384 |
| ernie-3.0-nano-zh | Chinese | Layer:4, Heads:12, Hidden:312 |
| ernie-health-zh | Chinese | Layer:12, Heads:12, Hidden:768 |
| ernie-gram-zh | Chinese | Layer:12, Heads:12, Hidden:768 |
You can find all the supported models from huggingface's model hub: huggingface.co/nghuyong, and model details from paddle's official repo: PaddleNLP and ERNIE.
Documentation resources
- Text classification task guide
- Token classification task guide
- Question answering task guide
- Causal language modeling task guide
- Masked language modeling task guide
- Multiple choice task guide
ErnieConfig
autodoc ErnieConfig - all
Ernie specific outputs
autodoc models.ernie.modeling_ernie.ErnieForPreTrainingOutput
ErnieModel
autodoc ErnieModel - forward
ErnieForPreTraining
autodoc ErnieForPreTraining - forward
ErnieForCausalLM
autodoc ErnieForCausalLM - forward
ErnieForMaskedLM
autodoc ErnieForMaskedLM - forward
ErnieForNextSentencePrediction
autodoc ErnieForNextSentencePrediction - forward
ErnieForSequenceClassification
autodoc ErnieForSequenceClassification - forward
ErnieForMultipleChoice
autodoc ErnieForMultipleChoice - forward
ErnieForTokenClassification
autodoc ErnieForTokenClassification - forward
ErnieForQuestionAnswering
autodoc ErnieForQuestionAnswering - forward