mlc-llm icon indicating copy to clipboard operation
mlc-llm copied to clipboard

[Refactor] PagedKVCache spec for MLC-LLM

Open annanyapr opened this issue 6 months ago • 0 comments

In this PR I have used the new spec for PagedKVCache for mlc-llm model definition. I have also removed the usage of the MLC-LLM PagedKVCache and relied on the TVM version everywhere. We don't need to change much as the new PagedKVCache spec is the child of the Object spec so all the earlier implementation is still valid. This PR should be merged only after https://github.com/apache/tvm/pull/17837 is merged.

annanyapr avatar Apr 14 '25 16:04 annanyapr