transformers/docs/source
Younes Belkada fdb85be40f
Faster generation using AWQ + Fused modules (#27411)
* v1 fusing modules

* add fused mlp support

* up

* fix CI

* block save_pretrained

* fixup

* small fix

* add new condition

* add v1 docs

* add some comments

* style

* fix nit

* adapt from suggestion

* add check

* change arg names

* change variables name

* Update src/transformers/integrations/awq.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* style

* split up into 3 different private methods

* more conditions

* more checks

* add fused tests for custom models

* fix

* fix tests

* final update docs

* final fixes

* fix importlib metadata

* Update src/transformers/utils/quantization_config.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* change it to `do_fuse`

* nit

* Update src/transformers/utils/quantization_config.py

Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com>

* Update src/transformers/utils/quantization_config.py

Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com>

* Update src/transformers/utils/quantization_config.py

Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com>

* few fixes

* revert

* fix test

* fix copies

* raise error if model is not quantized

* add test

* use quantization_config.config when fusing

* Update src/transformers/modeling_utils.py

---------

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
Co-authored-by: Marc Sun <57196510+SunMarc@users.noreply.github.com>
2023-12-05 12:14:45 +01:00
..
de docs: replace torch.distributed.run by torchrun (#27528) 2023-11-27 16:26:33 +00:00
en Faster generation using AWQ + Fused modules (#27411) 2023-12-05 12:14:45 +01:00
es docs: replace torch.distributed.run by torchrun (#27528) 2023-11-27 16:26:33 +00:00
fr [i18n-fr] Translate installation to French (#27657) 2023-12-01 14:00:07 +01:00
hi Hindi translation of pipeline_tutorial.md (#26837) 2023-10-25 11:21:49 -07:00
it docs: replace torch.distributed.run by torchrun (#27528) 2023-11-27 16:26:33 +00:00
ja Translate en/tasks folder docs to Japanese 🇯🇵 (#27098) 2023-12-04 14:10:54 -08:00
ko docs: replace torch.distributed.run by torchrun (#27528) 2023-11-27 16:26:33 +00:00
ms TVP model (#25856) 2023-11-21 16:41:55 +00:00
pt docs: replace torch.distributed.run by torchrun (#27528) 2023-11-27 16:26:33 +00:00
te Added Telugu [te] translations (#26828) 2023-10-20 15:27:55 -07:00
tr Translate index.md to Turkish (#27093) 2023-11-08 08:35:20 -05:00
zh translate internal folder files to chinese (#27638) 2023-12-04 10:04:28 -08:00
_config.py [Styling] stylify using ruff (#27144) 2023-11-16 17:43:19 +01:00