lightllm icon indicating copy to clipboard operation
lightllm copied to clipboard
verified publisher icon ModelTC

[Feature]框架支持prefix cache吗

Open inkhare opened this issue 9 months ago • 3 comments

inkhare avatar Mar 14 '25 16:03 inkhare

你说的是 radix cache这种技术吗。

hiworldwzj avatar Mar 17 '25 05:03 hiworldwzj

你说的是 radix cache这种技术吗。

是的

inkhare avatar Mar 17 '25 07:03 inkhare

https://github.com/ModelTC/lightllm/blob/main/lightllm/server/api_cli.py#L167

hiworldwzj avatar Mar 17 '25 08:03 hiworldwzj