riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-08-03 03:31:05 +06:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

Correctly raise error for awq quantization (#38945)

Browse Source 
fix warning
This commit is contained in:
Cyril Vallez 2025-06-20 17:18:06 +02:00 committed by GitHub
parent 8ac2d75353
commit 5ee60f970a
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 3 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
src/transformers/quantizers/quantizer_awq.py
View File

@ -82,7 +82,9 @@ class AwqQuantizer(HfQuantizer):
"your model on a GPU device in order to run your model."
)
elif device_map is not None:
if isinstance(device_map, dict) and ("cpu" in device_map.values() or "disk" in device_map.values()):
if isinstance(device_map, dict) and any(
forbidden in device_map.values() for forbidden in ("cpu", torch.device("cpu"), "disk")
):
raise ValueError(
"You are attempting to load an AWQ model with a device_map that contains a CPU or disk device."
" This is not supported. Please remove the CPU or disk device from the device_map."