riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-03 12:50:06 +06:00
Code Issues Actions 28 Packages Projects Releases Wiki Activity

[docs] Format fix (#38414)
Some checks are pending
Self-hosted runner (benchmark) / Benchmark (aws-g5-4xlarge-cache) (push) Waiting to run
Details
Build documentation / build (push) Waiting to run
Details
Slow tests on important models (on Push - A10) / Get all modified files (push) Waiting to run
Details
Slow tests on important models (on Push - A10) / Slow & FA2 tests (push) Blocked by required conditions
Details
Secret Leaks / trufflehog (push) Waiting to run
Details
Update Transformers metadata / build_and_package (push) Waiting to run
Details

Browse Source 
fix table
This commit is contained in:
Steven Liu 2025-06-03 09:53:23 -07:00 committed by GitHub
parent 0f41c41a46
commit 78d771c3c2
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 4 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
docs/source/en/cache_explanation.md
View File

@ -56,10 +56,10 @@ Attention is calculated independently in each layer of the model, and caching is
Refer to the table below to compare how caching improves efficiency.
| without caching | with caching | | | |
|---|---|---|---|---|
| for each step, recompute all previous `K` and `V` | for each step, only compute current `K` and `V` | | | |
| attention cost per step is **quadratic** with sequence length | attention cost per step is **linear** with sequence length (memory grows linearly, but compute/token remains low) | | | |
| without caching | with caching |
|---|---|
| for each step, recompute all previous `K` and `V` | for each step, only compute current `K` and `V`
| attention cost per step is **quadratic** with sequence length | attention cost per step is **linear** with sequence length (memory grows linearly, but compute/token remains low) |