61 Commits

Author	SHA1	Message	Date
Yih-Dar	c3ecf2d95d	Fix doctest CI (#26324) ... fix doc CI Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>	2023-09-22 08:58:30 +02:00
Yih-Dar	f02b915ba2	Remove utils/documentation_tests.txt (#26213) ... * update * update --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>	2023-09-18 13:33:01 +02:00
Yih-Dar	d020a2b81b	No doctest for convert_bros_to_pytorch.py (#26212) ... fix Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>	2023-09-18 13:31:59 +02:00
Arthur	9cccb3a838	[Persimmon] Add support for persimmon (#26042) ... * intiial commit * updates * nits * update conversion script * update conversion script * use path to load * add tips etc * some modeling logic * modeling update * more nits * nits * normal layer norm * update config and doc * nits * update doc remove unused * update * fix inits and stuff * fixup * revert wrong changes * updates * more nits * add default config values to the configuration file * fixup happy * update * 2 tests left * update readmes * more nits * slow test and more documentation * update readme * fix licences * styling * use fast if possible when saving tokenizer * remove todo * remove tokenization tests * small last nits * Apply suggestions from code review Co-authored-by: Matt <Rocketknight1@users.noreply.github.com> * nits to skip the timout doctest * fix integration test * fix test * update eos token * update to allow fast tokenization * styling * fix codeLlama as well for the update post processor * Apply suggestions from code review Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com> * add more copied from statements * update * doc passes doctest * remove `# final layer norm?` * change docstring prompot * update * Update README.md Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> * don't doctest the conversion script as it requires more packages * don't init a model in the config * oups * fix doctest --------- Co-authored-by: Matt <Rocketknight1@users.noreply.github.com> Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com> Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>	2023-09-12 11:33:27 +02:00
Joao Gante	a35f889acc	Tests: detect lines removed from "utils/not_doctested.txt" and doctest ALL generation files (#25763)	2023-08-29 16:15:05 +01:00
Younes Belkada	4b79697865	🚨🚨🚨 [Refactor] Move third-party related utility files into integrations/ folder 🚨🚨🚨 (#25599) ... * move deepspeed to `lib_integrations.deepspeed` * more refactor * oops * fix slow tests * Fix docs * fix docs * addess feedback * address feedback * final modifs for PEFT * fixup * ok now * trigger CI * trigger CI again * Update docs/source/en/main_classes/deepspeed.md Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> * import from `integrations` * address feedback * revert removal of `deepspeed` module * revert removal of `deepspeed` module * fix conflicts * ooops * oops * add deprecation warning * place it on the top * put `FutureWarning` * fix conflicts with not_doctested.txt * add back `bitsandbytes` module with a depr warning * fix * fix * fixup * oops * fix doctests --------- Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>	2023-08-25 17:13:34 +02:00
Joao Gante	494e96d8d6	Generate: logits processors are doctested and fix broken doctests (#25692) ... * shorter example * add logits processors to doctests * remove file from conflict? * tmp commit * Fix broken tests; Shorter sampling tests * Apply suggestions from code review Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com> --------- Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>	2023-08-25 12:42:06 +01:00
Yih-Dar	b413e0610b	Remove utils/documentation_tests.txt (#25680) ... * fix * fix * fix * fix * fix * fix * Apply suggestions from code review Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com> --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>	2023-08-23 11:14:45 +02:00
Yih-Dar	3d1edb6c5d	fix wrong path in some doc (#25658) ... * update * check --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>	2023-08-23 08:34:30 +02:00
Yih-Dar	2df24228d6	Skip doctest for some recent files (#25631) ... update Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>	2023-08-21 15:20:44 +02:00
Yih-Dar	f09db47a71	Run doctest for new files (#25588) ... fix Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>	2023-08-21 11:08:38 +02:00