Niklas Muennighoff ecd61c6286 Add OLMoE (#32406) ... * Add OLMoE * Add OLMoE * Updates * Make norm optional; add keys * Add output * Add * Fix dtype * Fix eos config * Update * Add OLMoE * Fix OLMoE path * Format * Format * Rmv copy statement * Rmv copy statement * Format * Add copies * Cp rotary * Fix aming * Fix naming * Update RoPE integration; num_logits_to_keep; Add copy statements * Add eps to config * Format * Add aux loss * Adapt router_aux_loss_coef * Update md * Adapt * adapt tests		2024-09-03 18:43:12 +02:00
..
__init__.py	Add OLMoE (#32406)	2024-09-03 18:43:12 +02:00
test_modeling_olmoe.py	Add OLMoE (#32406)	2024-09-03 18:43:12 +02:00