riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-04 05:10:06 +06:00
Code Issues Actions 26 Packages Projects Releases Wiki Activity

Fixing Bitnet after use_rms_norm introduction (#38229)

Browse Source 
* fix

* make style
This commit is contained in:
Mohamed Mekkouri 2025-05-20 17:13:21 +02:00 committed by GitHub
parent 9661896083
commit 2a79471318
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
src/transformers/integrations/bitnet.py
View File

@ -362,8 +362,8 @@ def _replace_with_bitnet_linear(
bias=module.bias is not None,
device=module.weight.device,
dtype=module.weight.dtype,
use_rms_norm=quantization_config.use_rms_norm,
rms_norm_eps=quantization_config.rms_norm_eps,
use_rms_norm=quantization_config.use_rms_norm if quantization_config else False,
rms_norm_eps=quantization_config.rms_norm_eps if quantization_config else 1e-6,
)
model._modules[name].requires_grad_(False)
has_been_replaced = True