mirror of
https://github.com/huggingface/transformers.git
synced 2025-07-06 06:10:04 +06:00
81 lines
1.8 KiB
ReStructuredText
81 lines
1.8 KiB
ReStructuredText
RoBERTa
|
|
----------------------------------------------------
|
|
|
|
The RoBERTa model was proposed in `RoBERTa: A Robustly Optimized BERT Pretraining Approach`_
|
|
by Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer,
|
|
Veselin Stoyanov. It is based on Google's BERT model released in 2018.
|
|
|
|
It builds on BERT and modifies key hyperparameters, removing the next-sentence pretraining
|
|
objective and training with much larger mini-batches and learning rates.
|
|
|
|
This implementation is the same as BertModel with a tiny embeddings tweak as well as a setup for Roberta pretrained
|
|
models.
|
|
|
|
RobertaConfig
|
|
~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
.. autoclass:: transformers.RobertaConfig
|
|
:members:
|
|
|
|
|
|
RobertaTokenizer
|
|
~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
.. autoclass:: transformers.RobertaTokenizer
|
|
:members:
|
|
|
|
|
|
RobertaModel
|
|
~~~~~~~~~~~~~~~~~~~~
|
|
|
|
.. autoclass:: transformers.RobertaModel
|
|
:members:
|
|
|
|
|
|
RobertaForMaskedLM
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
.. autoclass:: transformers.RobertaForMaskedLM
|
|
:members:
|
|
|
|
|
|
RobertaForSequenceClassification
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
.. autoclass:: transformers.RobertaForSequenceClassification
|
|
:members:
|
|
|
|
|
|
RobertaForTokenClassification
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
.. autoclass:: transformers.RobertaForTokenClassification
|
|
:members:
|
|
|
|
TFRobertaModel
|
|
~~~~~~~~~~~~~~~~~~~~
|
|
|
|
.. autoclass:: transformers.TFRobertaModel
|
|
:members:
|
|
|
|
|
|
TFRobertaForMaskedLM
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
.. autoclass:: transformers.TFRobertaForMaskedLM
|
|
:members:
|
|
|
|
|
|
TFRobertaForSequenceClassification
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
.. autoclass:: transformers.TFRobertaForSequenceClassification
|
|
:members:
|
|
|
|
|
|
TFRobertaForTokenClassification
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
.. autoclass:: transformers.TFRobertaForTokenClassification
|
|
:members:
|