Teng Ma
Teng Ma
https://github.com/kvcache-ai/Mooncake/blob/main/doc/en/mooncake-store.md#mooncake-store-python-api
Another example is deployed with hicache and sglang. https://github.com/sgl-project/sglang/blob/main/python/sglang/srt/mem_cache/storage/mooncake_store/README.md
> 1.#Start the mooncake main service (default port 50051) and the HTTP metadata service (default port 8080) (MooncakeStore_venv) root@ubuntu:/workspaces/zhangjh/MooncakeStore# cat Mooncake_master.sh mooncake_master --rpc_port 50051 --metrics_port 9003 --enable_metric_reporting 1 --enable_http_metadata_server 1...
@zhongzhouTan-coder @nickyc975 We should consider this case as well. #607
> Sure, this can be an optimization case such as `graceful shutdown`, we can consider it after the `Phase 1 And Phase 2` complete. For `Grace shutdown`, we can migrate...
> Why is getBatchTransferStatus included in the API getTransferStatus? > > TE Transfer is a batch-based API, so we need to retrieve the status of the entire batch.
> Celebrate! Finally, we have the V3 Roadmap! I have a few questions I'd like to ask: > > 1. What is the motivation of 'key-based routing'? > 2. Have...
> Congratulations, just a big v3 roadmap!!!! I am very interesting in the Cache Scheduling such as hot data migration, I hope I can join the contribution and do some...
👍 I will review it today.
> 感谢你的解答！ 今天我们在线下讨论时，对SSD存储的使用有一定困惑：对于存放在SSD的kvcache，应该在什么时机取回DRAM？ 如果是在命中时取回，IO+通信耗时可能要大于重计算的耗时。请问你是如何看待这个问题的？ 需要看场景，如果input很长，prefetch kvcache是有意义的