mirror of
https://github.com/huggingface/transformers.git
synced 2025-07-05 22:00:09 +06:00
eedd21b9e7
* Fixed typo: insted to instead * Fixed typo: relase to release * Fixed typo: nighlty to nightly * Fixed typos: versatible, benchamarks, becnhmark to versatile, benchmark, benchmarks * Fixed typo in comment: quantizd to quantized * Fixed typo: architecutre to architecture * Fixed typo: contibution to contribution * Fixed typo: Presequities to Prerequisites * Fixed typo: faste to faster * Fixed typo: extendeding to extending * Fixed typo: segmetantion_maps to segmentation_maps * Fixed typo: Alternativelly to Alternatively * Fixed incorrectly defined variable: output to output_disabled * Fixed typo in library name: tranformers.onnx to transformers.onnx * Fixed missing import: import tensorflow as tf * Fixed incorrectly defined variable: token_tensor to tokens_tensor * Fixed missing import: import torch * Fixed incorrectly defined variable and typo: uromaize to uromanize * Fixed incorrectly defined variable and typo: uromaize to uromanize * Fixed typo in function args: numpy.ndarry to numpy.ndarray * Fixed Inconsistent Library Name: Torchscript to TorchScript * Fixed Inconsistent Class Name: OneformerProcessor to OneFormerProcessor * Fixed Inconsistent Class Named Typo: TFLNetForMultipleChoice to TFXLNetForMultipleChoice * Fixed Inconsistent Library Name Typo: Pytorch to PyTorch * Fixed Inconsistent Function Name Typo: captureWarning to captureWarnings * Fixed Inconsistent Library Name Typo: Pytorch to PyTorch * Fixed Inconsistent Class Name Typo: TrainingArgument to TrainingArguments * Fixed Inconsistent Model Name Typo: Swin2R to Swin2SR * Fixed Inconsistent Model Name Typo: EART to BERT * Fixed Inconsistent Library Name Typo: TensorFLow to TensorFlow * Fixed Broken Link for Speech Emotion Classification with Wav2Vec2 * Fixed minor missing word Typo * Fixed minor missing word Typo * Fixed minor missing word Typo * Fixed minor missing word Typo * Fixed minor missing word Typo * Fixed minor missing word Typo * Fixed minor missing word Typo * Fixed minor missing word Typo * Fixed Punctuation: Two commas * Fixed Punctuation: No Space between XLM-R and is * Fixed Punctuation: No Space between [~accelerate.Accelerator.backward] and method * Added backticks to display model.fit() in codeblock * Added backticks to display openai-community/gpt2 in codeblock * Fixed Minor Typo: will to with * Fixed Minor Typo: is to are * Fixed Minor Typo: in to on * Fixed Minor Typo: inhibits to exhibits * Fixed Minor Typo: they need to it needs * Fixed Minor Typo: cast the load the checkpoints To load the checkpoints * Fixed Inconsistent Class Name Typo: TFCamembertForCasualLM to TFCamembertForCausalLM * Fixed typo in attribute name: outputs.last_hidden_states to outputs.last_hidden_state * Added missing verbosity level: fatal * Fixed Minor Typo: take To takes * Fixed Minor Typo: heuristic To heuristics * Fixed Minor Typo: setting To settings * Fixed Minor Typo: Content To Contents * Fixed Minor Typo: millions To million * Fixed Minor Typo: difference To differences * Fixed Minor Typo: while extract To which extracts * Fixed Minor Typo: Hereby To Here * Fixed Minor Typo: addition To additional * Fixed Minor Typo: supports To supported * Fixed Minor Typo: so that benchmark results TO as a consequence, benchmark * Fixed Minor Typo: a To an * Fixed Minor Typo: a To an * Fixed Minor Typo: Chain-of-though To Chain-of-thought
136 lines
4.2 KiB
Markdown
136 lines
4.2 KiB
Markdown
<!--Copyright 2020 The HuggingFace Team. All rights reserved.
|
|
|
|
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
|
the License. You may obtain a copy of the License at
|
|
|
|
http://www.apache.org/licenses/LICENSE-2.0
|
|
|
|
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
|
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
specific language governing permissions and limitations under the License.
|
|
|
|
⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
|
|
rendered properly in your Markdown viewer.
|
|
|
|
-->
|
|
|
|
# CamemBERT
|
|
|
|
## Overview
|
|
|
|
The CamemBERT model was proposed in [CamemBERT: a Tasty French Language Model](https://arxiv.org/abs/1911.03894) by
|
|
[Louis Martin](https://huggingface.co/louismartin), [Benjamin Muller](https://huggingface.co/benjamin-mlr), [Pedro Javier Ortiz Suárez](https://huggingface.co/pjox), Yoann Dupont, Laurent Romary, Éric Villemonte de la
|
|
Clergerie, [Djamé Seddah](https://huggingface.co/Djame), and [Benoît Sagot](https://huggingface.co/sagot). It is based on Facebook's RoBERTa model released in 2019. It is a model
|
|
trained on 138GB of French text.
|
|
|
|
The abstract from the paper is the following:
|
|
|
|
*Pretrained language models are now ubiquitous in Natural Language Processing. Despite their success, most available
|
|
models have either been trained on English data or on the concatenation of data in multiple languages. This makes
|
|
practical use of such models --in all languages except English-- very limited. Aiming to address this issue for French,
|
|
we release CamemBERT, a French version of the Bi-directional Encoders for Transformers (BERT). We measure the
|
|
performance of CamemBERT compared to multilingual models in multiple downstream tasks, namely part-of-speech tagging,
|
|
dependency parsing, named-entity recognition, and natural language inference. CamemBERT improves the state of the art
|
|
for most of the tasks considered. We release the pretrained model for CamemBERT hoping to foster research and
|
|
downstream applications for French NLP.*
|
|
|
|
This model was contributed by [the ALMAnaCH team (Inria)](https://huggingface.co/almanach). The original code can be found [here](https://camembert-model.fr/).
|
|
|
|
<Tip>
|
|
|
|
This implementation is the same as RoBERTa. Refer to the [documentation of RoBERTa](roberta) for usage examples as well
|
|
as the information relative to the inputs and outputs.
|
|
|
|
</Tip>
|
|
|
|
## Resources
|
|
|
|
- [Text classification task guide](../tasks/sequence_classification)
|
|
- [Token classification task guide](../tasks/token_classification)
|
|
- [Question answering task guide](../tasks/question_answering)
|
|
- [Causal language modeling task guide](../tasks/language_modeling)
|
|
- [Masked language modeling task guide](../tasks/masked_language_modeling)
|
|
- [Multiple choice task guide](../tasks/multiple_choice)
|
|
|
|
## CamembertConfig
|
|
|
|
[[autodoc]] CamembertConfig
|
|
|
|
## CamembertTokenizer
|
|
|
|
[[autodoc]] CamembertTokenizer
|
|
- build_inputs_with_special_tokens
|
|
- get_special_tokens_mask
|
|
- create_token_type_ids_from_sequences
|
|
- save_vocabulary
|
|
|
|
## CamembertTokenizerFast
|
|
|
|
[[autodoc]] CamembertTokenizerFast
|
|
|
|
<frameworkcontent>
|
|
<pt>
|
|
|
|
## CamembertModel
|
|
|
|
[[autodoc]] CamembertModel
|
|
|
|
## CamembertForCausalLM
|
|
|
|
[[autodoc]] CamembertForCausalLM
|
|
|
|
## CamembertForMaskedLM
|
|
|
|
[[autodoc]] CamembertForMaskedLM
|
|
|
|
## CamembertForSequenceClassification
|
|
|
|
[[autodoc]] CamembertForSequenceClassification
|
|
|
|
## CamembertForMultipleChoice
|
|
|
|
[[autodoc]] CamembertForMultipleChoice
|
|
|
|
## CamembertForTokenClassification
|
|
|
|
[[autodoc]] CamembertForTokenClassification
|
|
|
|
## CamembertForQuestionAnswering
|
|
|
|
[[autodoc]] CamembertForQuestionAnswering
|
|
|
|
</pt>
|
|
<tf>
|
|
|
|
## TFCamembertModel
|
|
|
|
[[autodoc]] TFCamembertModel
|
|
|
|
## TFCamembertForCausalLM
|
|
|
|
[[autodoc]] TFCamembertForCausalLM
|
|
|
|
## TFCamembertForMaskedLM
|
|
|
|
[[autodoc]] TFCamembertForMaskedLM
|
|
|
|
## TFCamembertForSequenceClassification
|
|
|
|
[[autodoc]] TFCamembertForSequenceClassification
|
|
|
|
## TFCamembertForMultipleChoice
|
|
|
|
[[autodoc]] TFCamembertForMultipleChoice
|
|
|
|
## TFCamembertForTokenClassification
|
|
|
|
[[autodoc]] TFCamembertForTokenClassification
|
|
|
|
## TFCamembertForQuestionAnswering
|
|
|
|
[[autodoc]] TFCamembertForQuestionAnswering
|
|
|
|
</tf>
|
|
</frameworkcontent>
|
|
|