2 Commits

Author	SHA1	Message	Date
Sylvain Gugger	9d14be5c20	Add support for ZeRO-2/3 and ZeRO-offload in fairscale (#10354) ... * Ass support for ZeRO-2/3 and ZeRO-offload in fairscale * Quality * Rework from review comments * Add doc * Apply suggestions from code review Co-authored-by: Stas Bekman <stas00@users.noreply.github.com> * Address review comments Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>	2021-02-25 11:07:53 -05:00
Stas Bekman	0b1f552a24	fix run_seq2seq.py; porting trainer tests to it (#10162) ... * fix run_seq2seq.py; porting DeepSpeed tests to it * unrefactor * defensive programming * defensive programming 2 * port the rest of the trainer tests * style * a cleaner scripts dir finder * cleanup	2021-02-15 09:12:17 -08:00