4 Commits

Author	SHA1	Message	Date
Marc Sun	67890de3b8	Torchao weights only + prequantized compability (#34355) ... * weights only compability * better tests from code review * ping torch version * add weights_only check	2024-11-20 17:24:45 +01:00
Benjamin Bossan	5e1fd4e204	FIX: Broken repr of TorchAoConfig (#34560) ... FIX Broken repr of TorchAoConfig The __repr__ method references a non-existent self.kwargs. This is now fixed. There does not appear to be a uniform way of defining __repr__ for quantization configs. I copied the method as implemented for HQQ: e2ac16b28a/src/transformers/utils/quantization_config.py (L285-L287)	2024-11-05 10:26:13 +01:00
Marc Sun	004530aa05	Fix regression loading dtype (#34409) ... * fix regression * add test for torchao * expected output * better fix	2024-10-29 11:41:04 +01:00
Jerry Zhang	78d78cdf8a	Add TorchAOHfQuantizer (#32306) ... * Add TorchAOHfQuantizer Summary: Enable loading torchao quantized model in huggingface. Test Plan: local test Reviewers: Subscribers: Tasks: Tags: * Fix a few issues * style * Added tests and addressed some comments about dtype conversion * fix torch_dtype warning message * fix tests * style * TorchAOConfig -> TorchAoConfig * enable offload + fix memory with multi-gpu * update torchao version requirement to 0.4.0 * better comments * add torch.compile to torchao README, add perf number link --------- Co-authored-by: Marc Sun <marc@huggingface.co>	2024-08-14 16:14:24 +02:00