mirror of
https://github.com/huggingface/transformers.git
synced 2025-07-24 23:08:57 +06:00
86777b5e2f
* Support `AOPerModuleConfig` and include_embedding Summary: This PR adds support per module configuration for torchao Also added per module quantization examples: 1. Quantizing different layers with different quantization configs 2. Skip quantization for certain layers Test Plan: python tests/quantization/torchao_integration/test_torchao.py -k test_include_embedding python tests/quantization/torchao_integration/test_torchao.py -k test_per_module_config_skip Reviewers: Subscribers: Tasks: Tags: * format * format * inlcude embedding remove input embedding from module not to convert * more docs * Update docs/source/en/quantization/torchao.md Co-authored-by: Mohamed Mekkouri <93391238+MekkCyber@users.noreply.github.com> * Update src/transformers/quantizers/quantizer_torchao.py Co-authored-by: Mohamed Mekkouri <93391238+MekkCyber@users.noreply.github.com> * Update src/transformers/quantizers/quantizer_torchao.py Co-authored-by: Mohamed Mekkouri <93391238+MekkCyber@users.noreply.github.com> --------- Co-authored-by: Mohamed Mekkouri <93391238+MekkCyber@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| aqlm_integration | ||
| autoawq | ||
| autoround | ||
| bitnet_integration | ||
| bnb | ||
| compressed_tensors_integration | ||
| eetq_integration | ||
| fbgemm_fp8 | ||
| finegrained_fp8 | ||
| ggml | ||
| gptq | ||
| higgs | ||
| hqq | ||
| quanto_integration | ||
| quark_integration | ||
| spqr_integration | ||
| torchao_integration | ||
| vptq_integration | ||