Inference for multilingual models (#15836)

* 📝 first draft for multilingual models

* 🖍 make style

docs/source/_toctree.yml

@@ -35,6 +35,8 @@
 - sections:
   - local: create_a_model
     title: Create a custom model
+  - local: multilingual
+    title: Inference for multilingual models
   - local: examples
     title: Examples
   - local: troubleshooting

docs/source/multilingual.mdx

@@ -1,4 +1,4 @@
-<!--Copyright 2020 The HuggingFace Team. All rights reserved.
+<!--Copyright 2022 The HuggingFace Team. All rights reserved.
 
 Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 the License. You may obtain a copy of the License at
@@ -10,38 +10,31 @@ an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express o
 specific language governing permissions and limitations under the License.
 -->
 
-# Multi-lingual models
+# Multilingual models for inference
 
 [[open-in-colab]]
 
-Most of the models available in this library are mono-lingual models (English, Chinese and German). A few multi-lingual
-models are available and have a different mechanisms than mono-lingual models. This page details the usage of these
-models.
+There are several multilingual models in 🤗 Transformers, and their inference usage differs from monolingual models. Not *all* multilingual model usage is different though. Some models, like [bert-base-multilingual-uncased](https://huggingface.co/bert-base-multilingual-uncased), can be used just like a monolingual model. This guide will show you how to use multilingual models whose usage differs for inference.
 
 ## XLM
 
-XLM has a total of 10 different checkpoints, only one of which is mono-lingual. The 9 remaining model checkpoints can
-be split in two categories: the checkpoints that make use of language embeddings, and those that don't
+XLM has ten different checkpoints, only one of which is monolingual. The nine remaining model checkpoints can be split into two categories: the checkpoints that use language embeddings and those that don't.
 
-### XLM & Language Embeddings
+### XLM with language embeddings
 
-This section concerns the following checkpoints:
+The following XLM models use language embeddings to specify the language used at inference:
 
 - `xlm-mlm-ende-1024` (Masked language modeling, English-German)
 - `xlm-mlm-enfr-1024` (Masked language modeling, English-French)
 - `xlm-mlm-enro-1024` (Masked language modeling, English-Romanian)
 - `xlm-mlm-xnli15-1024` (Masked language modeling, XNLI languages)
-- `xlm-mlm-tlm-xnli15-1024` (Masked language modeling + Translation, XNLI languages)
+- `xlm-mlm-tlm-xnli15-1024` (Masked language modeling + translation, XNLI languages)
 - `xlm-clm-enfr-1024` (Causal language modeling, English-French)
 - `xlm-clm-ende-1024` (Causal language modeling, English-German)
 
-These checkpoints require language embeddings that will specify the language used at inference time. These language
-embeddings are represented as a tensor that is of the same shape as the input ids passed to the model. The values in
-these tensors depend on the language used and are identifiable using the `lang2id` and `id2lang` attributes from
-the tokenizer.
-
-Here is an example using the `xlm-clm-enfr-1024` checkpoint (Causal language modeling, English-French):
+Language embeddings are represented as a tensor of the same shape as the `input_ids` passed to the model. The values in these tensors depend on the language used and are identified by the tokenizer's `lang2id` and `id2lang` attributes.
 
+In this example, load the `xlm-clm-enfr-1024` checkpoint (Causal language modeling, English-French):
 
 ```py
 >>> import torch
@@ -51,22 +44,20 @@ Here is an example using the `xlm-clm-enfr-1024` checkpoint (Causal language mod
 >>> model = XLMWithLMHeadModel.from_pretrained("xlm-clm-enfr-1024")
 ```
 
-The different languages this model/tokenizer handles, as well as the ids of these languages are visible using the
-`lang2id` attribute:
+The `lang2id` attribute of the tokenizer displays this model's languages and their ids:
 
 ```py
 >>> print(tokenizer.lang2id)
 {'en': 0, 'fr': 1}
 ```
 
-These ids should be used when passing a language parameter during a model pass. Let's define our inputs:
+Next, create an example input:
 
 ```py
 >>> input_ids = torch.tensor([tokenizer.encode("Wikipedia was used to")])  # batch size of 1
 ```
 
-We should now define the language embedding by using the previously defined language id. We want to create a tensor
-filled with the appropriate language ids, of the same size as input_ids. For english, the id is 0:
+Set the language id as `"en"` and use it to define the language embedding. The language embedding is a tensor filled with `0` since that is the language id for English. This tensor should be the same size as `input_ids`. 
 
 ```py
 >>> language_id = tokenizer.lang2id["en"]  # 0
@@ -76,43 +67,109 @@ filled with the appropriate language ids, of the same size as input_ids. For eng
 >>> langs = langs.view(1, -1)  # is now of shape [1, sequence_length] (we have a batch size of 1)
 ```
 
-You can then feed it all as input to your model:
+Now you can pass the `input_ids` and language embedding to the model:
 
 ```py
 >>> outputs = model(input_ids, langs=langs)
 ```
 
-The example [run_generation.py](https://github.com/huggingface/transformers/tree/master/examples/pytorch/text-generation/run_generation.py) can generate text
-using the CLM checkpoints from XLM, using the language embeddings.
+The [run_generation.py](https://github.com/huggingface/transformers/tree/master/examples/pytorch/text-generation/run_generation.py) script can generate text with language embeddings using the `xlm-clm` checkpoints.
 
-### XLM without Language Embeddings
+### XLM without language embeddings
 
-This section concerns the following checkpoints:
+The following XLM models do not require language embeddings during inference:
 
 - `xlm-mlm-17-1280` (Masked language modeling, 17 languages)
 - `xlm-mlm-100-1280` (Masked language modeling, 100 languages)
 
-These checkpoints do not require language embeddings at inference time. These models are used to have generic sentence
-representations, differently from previously-mentioned XLM checkpoints.
-
+These models are used for generic sentence representations, unlike the previous XLM checkpoints.
 
 ## BERT
 
-BERT has two checkpoints that can be used for multi-lingual tasks:
+The following BERT models can be used for multilingual tasks:
 
 - `bert-base-multilingual-uncased` (Masked language modeling + Next sentence prediction, 102 languages)
 - `bert-base-multilingual-cased` (Masked language modeling + Next sentence prediction, 104 languages)
 
-These checkpoints do not require language embeddings at inference time. They should identify the language used in the
+These models do not require language embeddings during inference. They should identify the language from the
 context and infer accordingly.
 
 ## XLM-RoBERTa
 
-XLM-RoBERTa was trained on 2.5TB of newly created clean CommonCrawl data in 100 languages. It provides strong gains
-over previously released multi-lingual models like mBERT or XLM on downstream tasks like classification, sequence
-labeling and question answering.
-
-Two XLM-RoBERTa checkpoints can be used for multi-lingual tasks:
+The following XLM-RoBERTa models can be used for multilingual tasks:
 
 - `xlm-roberta-base` (Masked language modeling, 100 languages)
 - `xlm-roberta-large` (Masked language modeling, 100 languages)
+
+XLM-RoBERTa was trained on 2.5TB of newly created and cleaned CommonCrawl data in 100 languages. It provides strong gains over previously released multilingual models like mBERT or XLM on downstream tasks like classification, sequence labeling, and question answering.
+
+## M2M100
+
+The following M2M100 models can be used for multilingual translation:
+
+- `facebook/m2m100_418M` (Translation)
+- `facebook/m2m100_1.2B` (Translation)
+
+In this example, load the `facebook/m2m100_418M` checkpoint to translate from Chinese to English. You can set the source language in the tokenizer:
+
+```py
+>>> from transformers import M2M100ForConditionalGeneration, M2M100Tokenizer
+
+>>> en_text = "Do not meddle in the affairs of wizards, for they are subtle and quick to anger."
+>>> chinese_text = "不要插手巫師的事務, 因為他們是微妙的, 很快就會發怒."
+
+>>> tokenizer = M2M100Tokenizer.from_pretrained("facebook/m2m100_418M", src_lang="zh")
+>>> model = M2M100ForConditionalGeneration.from_pretrained("facebook/m2m100_418M")
+```
+
+Tokenize the text:
+
+```py
+>>> encoded_zh = tokenizer(chinese_text, return_tensors="pt")
+```
+
+M2M100 forces the target language id as the first generated token to translate to the target language. Set the `forced_bos_token_id` to `en` in the `generate` method to translate to English:
+
+```py
+>>> generated_tokens = model.generate(**encoded_zh, forced_bos_token_id=tokenizer.get_lang_id("en"))
+>>> tokenizer.batch_decode(generated_tokens, skip_special_tokens=True)
+'Do not interfere with the matters of the witches, because they are delicate and will soon be angry.'
+```
+
+## MBart
+
+The following MBart models can be used for multilingual translation:
+
+- `facebook/mbart-large-50-one-to-many-mmt` (One-to-many multilingual machine translation, 50 languages)
+- `facebook/mbart-large-50-many-to-many-mmt` (Many-to-many multilingual machine translation, 50 languages)
+- `facebook/mbart-large-50-many-to-one-mmt` (Many-to-one multilingual machine translation, 50 languages)
+- `facebook/mbart-large-50` (Multilingual translation, 50 languages)
+- `facebook/mbart-large-cc25`
+
+In this example, load the `facebook/mbart-large-50-many-to-many-mmt` checkpoint to translate Finnish to English. You can set the source language in the tokenizer:
+
+```py
+>>> from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
+
+>>> en_text = "Do not meddle in the affairs of wizards, for they are subtle and quick to anger."
+>>> fi_text = "Älä sekaannu velhojen asioihin, sillä ne ovat hienovaraisia ja nopeasti vihaisia."
+
+>>> tokenizer = AutoTokenizer.from_pretrained("facebook/mbart-large-50-many-to-many-mmt", src_lang="fi_FI")
+>>> model = AutoModelForSeq2SeqLM.from_pretrained("facebook/mbart-large-50-many-to-many-mmt")
+```
+
+Tokenize the text:
+
+```py
+>>> encoded_en = tokenizer(en_text, return_tensors="pt")
+```
+
+MBart forces the target language id as the first generated token to translate to the target language. Set the `forced_bos_token_id` to `en` in the `generate` method to translate to English:
+
+```py
+>>> generated_tokens = model.generate(**encoded_en, forced_bos_token_id=tokenizer.lang_code_to_id("en_XX"))
+>>> tokenizer.batch_decode(generated_tokens, skip_special_tokens=True)
+"Don't interfere with the wizard's affairs, because they are subtle, will soon get angry."
+```
+
+If you are using the `facebook/mbart-large-50-many-to-one-mmt` checkpoint, you don't need to force the target language id as the first generated token otherwise the usage is the same.