Arthur 07360b6c9c [Llama2] Add support for Llama 2 (#24891) ... * add llama * add other readmes * update padding id in readme * add link to paper * fix paths and tokenizer * more nits * styling * fit operation in 2 lines when possible * nits * Apply suggestions from code review Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> * add form * update reademe * update readme, we don't have a default pad token * update test and tokenization * LLaMA instead of Llama * nits * add expected text * add greeedy output * styling * Update src/transformers/models/llama/modeling_llama.py Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> * sequential device map * skip relevant changes --------- Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>		2023-07-18 15:18:31 -04:00
..
albert.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
align.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
altclip.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
audio-spectrogram-transformer.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
auto.md	Check all objects are equally in the main __init__ file (#24573)	2023-06-29 17:49:59 +02:00
autoformer.md	[Time-Series] Added blog-post to tips (#24482)	2023-07-03 10:07:25 +02:00
bark.md	Add bark (#24086)	2023-07-17 17:53:24 +01:00
bart.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
barthez.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
bartpho.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
beit.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
bert-generation.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
bert-japanese.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
bert.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
bertweet.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
big_bird.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
bigbird_pegasus.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
biogpt.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
bit.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
blenderbot-small.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
blenderbot.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
blip-2.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
blip.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
bloom.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
bort.md	Deprecate models (#24787)	2023-07-13 11:46:54 -04:00
bridgetower.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
byt5.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
camembert.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
canine.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
chinese_clip.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
clap.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
clip.md	Update old existing feature extractor references (#24552)	2023-06-29 10:17:36 +01:00
clipseg.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
codegen.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
conditional_detr.md	Removal of deprecated vision methods and specify deprecation versions (#24570)	2023-06-29 15:09:51 +01:00
convbert.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
convnext.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
convnextv2.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
cpm.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
cpmant.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
ctrl.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
cvt.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
data2vec.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
deberta-v2.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
deberta.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
decision_transformer.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
deformable_detr.md	Removal of deprecated vision methods and specify deprecation versions (#24570)	2023-06-29 15:09:51 +01:00
deit.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
deplot.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
deta.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
detr.md	Removal of deprecated vision methods and specify deprecation versions (#24570)	2023-06-29 15:09:51 +01:00
dialogpt.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
dinat.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
dinov2.md	Add DINOv2 (#24016)	2023-07-18 15:34:06 +01:00
distilbert.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
dit.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
donut.md	Update old existing feature extractor references (#24552)	2023-06-29 10:17:36 +01:00
dpr.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
dpt.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
efficientformer.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
efficientnet.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
electra.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
encodec.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
encoder-decoder.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
ernie_m.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
ernie.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
esm.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
flan-t5.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
flan-ul2.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
flaubert.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
flava.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
fnet.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
focalnet.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
fsmt.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
funnel.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
git.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
glpn.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
gpt_bigcode.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
gpt_neo.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
gpt_neox_japanese.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
gpt_neox.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
gpt-sw3.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
gpt2.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
gptj.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
gptsan-japanese.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
graphormer.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
groupvit.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
herbert.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
hubert.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
ibert.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
imagegpt.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
informer.md	[Time-Series] Added blog-post to tips (#24482)	2023-07-03 10:07:25 +02:00
instructblip.md	Add InstructBLIP (#23460)	2023-06-26 11:23:57 +02:00
jukebox.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
layoutlm.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
layoutlmv2.md	Update old existing feature extractor references (#24552)	2023-06-29 10:17:36 +01:00
layoutlmv3.md	Update old existing feature extractor references (#24552)	2023-06-29 10:17:36 +01:00
layoutxlm.md	Update old existing feature extractor references (#24552)	2023-06-29 10:17:36 +01:00
led.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
levit.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
lilt.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
llama.md	[Llama2] Add support for Llama 2 (#24891)	2023-07-18 15:18:31 -04:00
llama2.md	[Llama2] Add support for Llama 2 (#24891)	2023-07-18 15:18:31 -04:00
longformer.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
longt5.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
luke.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
lxmert.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
m2m_100.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
marian.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
markuplm.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
mask2former.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
maskformer.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
matcha.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
mbart.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
mctct.md	Deprecate models (#24787)	2023-07-13 11:46:54 -04:00
mega.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
megatron_gpt2.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
megatron-bert.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
mgp-str.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
mluke.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
mms.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
mobilebert.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
mobilenet_v1.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
mobilenet_v2.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
mobilevit.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
mobilevitv2.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
mpnet.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
mra.md	Add Multi Resolution Analysis (MRA) (New PR) (#24513)	2023-07-10 10:50:43 +01:00
mt5.md	[T5] Add T5ForQuestionAnswering and MT5ForQuestionAnswering (#24481)	2023-06-27 10:07:06 -04:00
musicgen.md	Add Musicgen (#24109)	2023-06-29 14:48:59 +01:00
mvp.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
nat.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
nezha.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
nllb-moe.md	[docs] Fix NLLB-MoE links (#24388)	2023-06-20 17:34:20 -07:00
nllb.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
nystromformer.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
oneformer.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
open-llama.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
openai-gpt.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
opt.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
owlvit.md	Update warning messages reffering to post_process_object_detection (#24649)	2023-07-04 16:47:57 -03:00
pegasus_x.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
pegasus.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
perceiver.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
phobert.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
pix2struct.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
plbart.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
poolformer.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
prophetnet.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
qdqbert.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
rag.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
realm.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
reformer.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
regnet.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
rembert.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
resnet.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
retribert.md	Deprecate models (#24787)	2023-07-13 11:46:54 -04:00
roberta-prelayernorm.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
roberta.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
roc_bert.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
roformer.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
rwkv.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
sam.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
segformer.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
sew-d.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
sew.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
speech_to_text_2.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
speech_to_text.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
speech-encoder-decoder.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
speecht5.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
splinter.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
squeezebert.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
swiftformer.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
swin.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
swin2sr.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
swinv2.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
switch_transformers.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
t5.md	[Umt5] Add google's umt5 to transformers (#24477)	2023-07-03 07:38:21 +02:00
t5v1.1.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
table-transformer.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
tapas.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
tapex.md	Deprecate models (#24787)	2023-07-13 11:46:54 -04:00
time_series_transformer.md	[Time-Series] Added blog-post to tips (#24482)	2023-07-03 10:07:25 +02:00
timesformer.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
trajectory_transformer.md	Deprecate models (#24787)	2023-07-13 11:46:54 -04:00
transfo-xl.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
trocr.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
tvlt.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
ul2.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
umt5.md	[MT5] Fix CONFIG_MAPPING issue leading it to load umt5 class (#24678)	2023-07-07 11:33:54 +09:00
unispeech-sat.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
unispeech.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
upernet.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
van.md	Deprecate models (#24787)	2023-07-13 11:46:54 -04:00
videomae.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
vilt.md	Update old existing feature extractor references (#24552)	2023-06-29 10:17:36 +01:00
vision-encoder-decoder.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
vision-text-dual-encoder.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
visual_bert.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
vit_hybrid.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
vit_mae.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
vit_msn.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
vit.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
vivit.md	Add ViViT (#22518)	2023-07-11 14:04:04 +01:00
wav2vec2_phoneme.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
wav2vec2-conformer.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
wav2vec2.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
wavlm.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
whisper.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
xclip.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
xglm.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
xlm-prophetnet.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
xlm-roberta-xl.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
xlm-roberta.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
xlm-v.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
xlm.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
xlnet.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
xls_r.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
xlsr_wav2vec2.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
xmod.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
yolos.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00
yoso.md	Migrate doc files to Markdown. (#24376)	2023-06-20 18:07:47 -04:00