## BERTweet [BERTweet](https://huggingface.co/papers/2005.10200) shares the same architecture as [BERT-base](./bert), but it’s pretrained like [RoBERTa](./roberta) on English Tweets. It performs really well on Tweet-related tasks like part-of-speech tagging, named entity recognition, and text classification. You can find all the original BERTweet checkpoints under the [VinAI Research](https://huggingface.co/vinai?search_models=BERTweet) organization. > [!TIP] > Refer to the [BERT](./bert) docs for more examples of how to apply BERTweet to different language tasks. The example below demonstrates how to predict the `` token with [`Pipeline`], [`AutoModel`], and from the command line. ```py import torch from transformers import pipeline pipeline = pipeline( task="fill-mask", model="vinai/bertweet-base", torch_dtype=torch.float16, device=0 ) pipeline("Plants create through a process known as photosynthesis.") ``` ```py import torch from transformers import AutoModelForMaskedLM, AutoTokenizer tokenizer = AutoTokenizer.from_pretrained( "vinai/bertweet-base", ) model = AutoModelForMaskedLM.from_pretrained( "vinai/bertweet-base", torch_dtype=torch.float16, device_map="auto" ) inputs = tokenizer("Plants create through a process known as photosynthesis.", return_tensors="pt").to("cuda") with torch.no_grad(): outputs = model(**inputs) predictions = outputs.logits masked_index = torch.where(inputs['input_ids'] == tokenizer.mask_token_id)[1] predicted_token_id = predictions[0, masked_index].argmax(dim=-1) predicted_token = tokenizer.decode(predicted_token_id) print(f"The predicted token is: {predicted_token}") ``` ```bash echo -e "Plants create through a process known as photosynthesis." | transformers-cli run --task fill-mask --model vinai/bertweet-base --device 0 ``` ## Notes - Use the [`AutoTokenizer`] or [`BertweetTokenizer`] because it’s preloaded with a custom vocabulary adapted to tweet-specific tokens like hashtags (#), mentions (@), emojis, and common abbreviations. Make sure to also install the [emoji](https://pypi.org/project/emoji/) library. - Inputs should be padded on the right (`padding="max_length"`) because BERT uses absolute position embeddings. ## BertweetTokenizer [[autodoc]] BertweetTokenizer