# XLM
[XLM](https://huggingface.co/papers/1901.07291) demonstrates cross-lingual pretraining with two approaches, unsupervised training on a single language and supervised training on more than one language with a cross-lingual language model objective. The XLM model supports the causal language modeling objective, masked language modeling, and translation language modeling (an extension of the [BERT](./bert)) masked language modeling objective to multiple language inputs).
You can find all the original XLM checkpoints under the [Facebook AI community](https://huggingface.co/FacebookAI?search_models=xlm-mlm) organization.
> [!TIP]
> Click on the XLM models in the right sidebar for more examples of how to apply XLM to different cross-lingual tasks like classification, translation, and question answering.
The example below demonstrates how to predict the `` token with [`Pipeline`], [`AutoModel`] and from the command line.
```python
import torch
from transformers import pipeline
pipeline = pipeline(
task="fill-mask",
model="facebook/xlm-roberta-xl",
torch_dtype=torch.float16,
device=0
)
pipeline("Bonjour, je suis un modèle .")
```
```python
import torch
from transformers import AutoModelForMaskedLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained(
"FacebookAI/xlm-mlm-en-2048",
)
model = AutoModelForMaskedLM.from_pretrained(
"FacebookAI/xlm-mlm-en-2048",
torch_dtype=torch.float16,
device_map="auto",
)
inputs = tokenizer("Hello, I'm a model.", return_tensors="pt").to("cuda")
with torch.no_grad():
outputs = model(**inputs)
predictions = outputs.logits.argmax(dim=-1)
predicted_token = tokenizer.decode(predictions[0][inputs["input_ids"][0] == tokenizer.mask_token_id])
print(f"Predicted token: {predicted_token}")
```
```bash
echo -e "Plants create through a process known as photosynthesis." | transformers-cli run --task fill-mask --model FacebookAI/xlm-mlm-en-2048 --device 0
```
## XLMConfig
[[autodoc]] XLMConfig
## XLMTokenizer
[[autodoc]] XLMTokenizer
- build_inputs_with_special_tokens
- get_special_tokens_mask
- create_token_type_ids_from_sequences
- save_vocabulary
## XLM specific outputs
[[autodoc]] models.xlm.modeling_xlm.XLMForQuestionAnsweringOutput
## XLMModel
[[autodoc]] XLMModel
- forward
## XLMWithLMHeadModel
[[autodoc]] XLMWithLMHeadModel
- forward
## XLMForSequenceClassification
[[autodoc]] XLMForSequenceClassification
- forward
## XLMForMultipleChoice
[[autodoc]] XLMForMultipleChoice
- forward
## XLMForTokenClassification
[[autodoc]] XLMForTokenClassification
- forward
## XLMForQuestionAnsweringSimple
[[autodoc]] XLMForQuestionAnsweringSimple
- forward
## XLMForQuestionAnswering
[[autodoc]] XLMForQuestionAnswering
- forward
## TFXLMModel
[[autodoc]] TFXLMModel
- call
## TFXLMWithLMHeadModel
[[autodoc]] TFXLMWithLMHeadModel
- call
## TFXLMForSequenceClassification
[[autodoc]] TFXLMForSequenceClassification
- call
## TFXLMForMultipleChoice
[[autodoc]] TFXLMForMultipleChoice
- call
## TFXLMForTokenClassification
[[autodoc]] TFXLMForTokenClassification
- call
## TFXLMForQuestionAnsweringSimple
[[autodoc]] TFXLMForQuestionAnsweringSimple
- call
