v6d Integrate the vineyard llm kv cache with vllm

Integrate the vineyard llm kv cache with vllm

Open dashanji opened this issue 11 months ago • 1 comments

Since we have the python API of llm kv cache, we can integrate it with vllm.

Mar 20 '24 05:03 dashanji

/cc @sighingnow, this issus/pr has had no activity for a long time, please help to review the status and assign people to work on it.

Apr 12 '24 00:04 github-actions[bot]

Closed as finished.

Jul 15 '24 15:07 dashanji