mirror of
https://github.com/huggingface/transformers.git
synced 2025-07-03 04:40:06 +06:00
797860c68c
Some checks are pending
Self-hosted runner (benchmark) / Benchmark (aws-g5-4xlarge-cache) (push) Waiting to run
Build documentation / build (push) Waiting to run
New model PR merged notification / Notify new model (push) Waiting to run
Slow tests on important models (on Push - A10) / Get all modified files (push) Waiting to run
Slow tests on important models (on Push - A10) / Slow & FA2 tests (push) Blocked by required conditions
Self-hosted runner (push-caller) / Check if setup was changed (push) Waiting to run
Self-hosted runner (push-caller) / build-docker-containers (push) Blocked by required conditions
Self-hosted runner (push-caller) / Trigger Push CI (push) Blocked by required conditions
Secret Leaks / trufflehog (push) Waiting to run
Update Transformers metadata / build_and_package (push) Waiting to run
* feat: add flexible Liger Kernel configuration to TrainingArguments
Add support for granular Liger Kernel configuration through a new
`liger_kernel_config` parameter in TrainingArguments. This allows users
to selectively enable/disable specific kernels (rope, swiglu, cross_entropy,
etc.) instead of the current approach that rely on default configuration.
Features:
- Add `liger_kernel_config` dict parameter to TrainingArguments
- Support selective kernel application for all supported models
- Maintain full backward compatibility with existing `use_liger_kernel` flag
Example usage:
```python
TrainingArguments(
use_liger_kernel=True,
liger_kernel_config={
"rope": True,
"swiglu": True,
"cross_entropy": False,
"fused_linear_cross_entropy": True
}
)
Closes #38905
* Address comments and update Liger section in Trainer docs
|
||
|---|---|---|
| .. | ||
| __init__.py | ||
| test_data_collator.py | ||
| test_trainer_callback.py | ||
| test_trainer_distributed_loss.py | ||
| test_trainer_distributed_worker_seed.py | ||
| test_trainer_distributed.py | ||
| test_trainer_fsdp.py | ||
| test_trainer_seq2seq.py | ||
| test_trainer_tpu.py | ||
| test_trainer_utils.py | ||
| test_trainer.py | ||