[docs] add the missing tokenizer when pushing models to huggingface hub (#33428)

* add tokenizer

* typo
This commit is contained in:
Fanli Lin 2024-09-12 00:56:55 +08:00 committed by GitHub
parent c403441339
commit cea9ec086a
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
2 changed files with 2 additions and 0 deletions

View File

@ -253,6 +253,7 @@ At this point, only three steps remain:
... train_dataset=lm_dataset["train"],
... eval_dataset=lm_dataset["test"],
... data_collator=data_collator,
... tokenizer=tokenizer,
... )
>>> trainer.train()

View File

@ -245,6 +245,7 @@ At this point, only three steps remain:
... train_dataset=lm_dataset["train"],
... eval_dataset=lm_dataset["test"],
... data_collator=data_collator,
... tokenizer=tokenizer,
... )
>>> trainer.train()