4 Commits

Author	SHA1	Message	Date
Joao Gante	7ffe25f2b9	Generate: end-to-end compilation (#30788) ... * mvp * added test (a few models need fixes) * fix a few test cases * test nits * harder test 😈 * revert changes in stablelm * test with improved condition * add todo * tmp commit * merged with main * nits * add todo * final corrections * add docs for generation compilation * docs nits * add tip * PR suggestions * add more details to the compilation docs * fix cache positions * cache is now init in generate; update docs * tag test as flaky * docs * post rebase make fixup and other nits * remove unintended changes * whisper (encoder-decoder) not supported * move token default updates to ; add tests for token defaults * push changes * manual rebase * chameleon doesn't support this * fix test_static_cache_mha_mqa_gqa (broken in another PR) * docs: dynamic is better with end-to-end compilation	2024-07-29 10:52:13 +01:00
Longjie Zheng	616bb11d48	Add torch.compile for Mistral (#30642) ... * first version * fix sliding window * fix style * add sliding window cache * fix style * address comments * fix test * fix style * move sliding window check inside cache init * revert changes on irrelevant files & add comment on SlidingWindowCache * address comments & fix style fix style * update causal mask * [run-slow] mistral * [run-slow] mistral * [run-slow] mistral * [run-slow] mistral * [run-slow] mistral * [run-slow] llama * [run-slow] mistral * [run-slow] mistral * [run-slow] mistral * revert CI from a10 to t4 * wrap up	2024-05-20 16:27:24 +02:00
Joao Gante	75bbfd5b22	Cache: Static cache as a standalone object (#30476)	2024-04-30 16:37:19 +01:00
Steven Liu	e74d793a3c	[docs] LLM inference (#29791) ... * first draft * feedback * static cache snippet * feedback * feedback	2024-04-22 12:41:51 -07:00