3 Commits

Author	SHA1	Message	Date
Longjie Zheng	616bb11d48	Add torch.compile for Mistral (#30642) ... * first version * fix sliding window * fix style * add sliding window cache * fix style * address comments * fix test * fix style * move sliding window check inside cache init * revert changes on irrelevant files & add comment on SlidingWindowCache * address comments & fix style fix style * update causal mask * [run-slow] mistral * [run-slow] mistral * [run-slow] mistral * [run-slow] mistral * [run-slow] mistral * [run-slow] llama * [run-slow] mistral * [run-slow] mistral * [run-slow] mistral * revert CI from a10 to t4 * wrap up	2024-05-20 16:27:24 +02:00
Joao Gante	75bbfd5b22	Cache: Static cache as a standalone object (#30476)	2024-04-30 16:37:19 +01:00
Steven Liu	e74d793a3c	[docs] LLM inference (#29791) ... * first draft * feedback * static cache snippet * feedback * feedback	2024-04-22 12:41:51 -07:00