mirror of
https://github.com/huggingface/transformers.git
synced 2025-07-06 14:20:04 +06:00
22f7218560
* add_ernie * remove Tokenizer in ernie * polish code * format code style * polish code * fix style * update doc * make fix-copies * change model name * change model name * fix dependency * add more copied from * rename ErnieLMHeadModel to ErnieForCausalLM do not expose ErnieLayer update doc * fix * make style * polish code * polish code * fix * fix * fix * fix * fix * final fix Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
102 lines
3.4 KiB
Plaintext
102 lines
3.4 KiB
Plaintext
<!--Copyright 2022 The HuggingFace Team. All rights reserved.
|
|
|
|
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
|
the License. You may obtain a copy of the License at
|
|
|
|
http://www.apache.org/licenses/LICENSE-2.0
|
|
|
|
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
|
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
specific language governing permissions and limitations under the License.
|
|
-->
|
|
|
|
# ERNIE
|
|
|
|
## Overview
|
|
ERNIE is a series of powerful models proposed by baidu, especially in Chinese tasks,
|
|
including [ERNIE1.0](https://arxiv.org/abs/1904.09223), [ERNIE2.0](https://ojs.aaai.org/index.php/AAAI/article/view/6428),
|
|
[ERNIE3.0](https://arxiv.org/abs/2107.02137), [ERNIE-Gram](https://arxiv.org/abs/2010.12148), [ERNIE-health](https://arxiv.org/abs/2110.07244), etc.
|
|
|
|
These models are contributed by [nghuyong](https://huggingface.co/nghuyong) and the official code can be found in [PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP) (in PaddlePaddle).
|
|
|
|
### How to use
|
|
Take `ernie-1.0-base-zh` as an example:
|
|
|
|
```Python
|
|
from transformers import AutoTokenizer, AutoModel
|
|
tokenizer = AutoTokenizer.from_pretrained("nghuyong/ernie-1.0-base-zh")
|
|
model = AutoModel.from_pretrained("nghuyong/ernie-1.0-base-zh")
|
|
```
|
|
|
|
### Supported Models
|
|
|
|
| Model Name | Language | Description |
|
|
|:-------------------:|:--------:|:-------------------------------:|
|
|
| ernie-1.0-base-zh | Chinese | Layer:12, Heads:12, Hidden:768 |
|
|
| ernie-2.0-base-en | English | Layer:12, Heads:12, Hidden:768 |
|
|
| ernie-2.0-large-en | English | Layer:24, Heads:16, Hidden:1024 |
|
|
| ernie-3.0-base-zh | Chinese | Layer:12, Heads:12, Hidden:768 |
|
|
| ernie-3.0-medium-zh | Chinese | Layer:6, Heads:12, Hidden:768 |
|
|
| ernie-3.0-mini-zh | Chinese | Layer:6, Heads:12, Hidden:384 |
|
|
| ernie-3.0-micro-zh | Chinese | Layer:4, Heads:12, Hidden:384 |
|
|
| ernie-3.0-nano-zh | Chinese | Layer:4, Heads:12, Hidden:312 |
|
|
| ernie-health-zh | Chinese | Layer:12, Heads:12, Hidden:768 |
|
|
| ernie-gram-zh | Chinese | Layer:12, Heads:12, Hidden:768 |
|
|
|
|
You can find all the supported models from huggingface's model hub: [huggingface.co/nghuyong](https://huggingface.co/nghuyong), and model details from paddle's official
|
|
repo: [PaddleNLP](https://paddlenlp.readthedocs.io/zh/latest/model_zoo/transformers/ERNIE/contents.html)
|
|
and [ERNIE](https://github.com/PaddlePaddle/ERNIE/blob/repro).
|
|
|
|
## ErnieConfig
|
|
|
|
[[autodoc]] ErnieConfig
|
|
- all
|
|
|
|
## Ernie specific outputs
|
|
|
|
[[autodoc]] models.ernie.modeling_ernie.ErnieForPreTrainingOutput
|
|
|
|
## ErnieModel
|
|
|
|
[[autodoc]] ErnieModel
|
|
- forward
|
|
|
|
## ErnieForPreTraining
|
|
|
|
[[autodoc]] ErnieForPreTraining
|
|
- forward
|
|
|
|
## ErnieForCausalLM
|
|
|
|
[[autodoc]] ErnieForCausalLM
|
|
- forward
|
|
|
|
## ErnieForMaskedLM
|
|
|
|
[[autodoc]] ErnieForMaskedLM
|
|
- forward
|
|
|
|
## ErnieForNextSentencePrediction
|
|
|
|
[[autodoc]] ErnieForNextSentencePrediction
|
|
- forward
|
|
|
|
## ErnieForSequenceClassification
|
|
|
|
[[autodoc]] ErnieForSequenceClassification
|
|
- forward
|
|
|
|
## ErnieForMultipleChoice
|
|
|
|
[[autodoc]] ErnieForMultipleChoice
|
|
- forward
|
|
|
|
## ErnieForTokenClassification
|
|
|
|
[[autodoc]] ErnieForTokenClassification
|
|
- forward
|
|
|
|
## ErnieForQuestionAnswering
|
|
|
|
[[autodoc]] ErnieForQuestionAnswering
|
|
- forward |