Liziqi-77 comments

Results 3 comments of


                                            Liziqi-77

[RFC]: Mooncake-Conductor: Design and Implementation of a Global Scheduler Module for KV-Cache-Centric Disaggregated Architecture

Currently, there are no histograms or other visualization functions for metrics such as kv cache hit rate. We believe it is necessary to add a module to Conductor to manage...

[Usage]: mem_cache_hit_nums and mem_cache_nums metrics not exposed via /metrics endpoint

> During validation, I noticed an issue: > > mem_cache_hit_nums_ increments every time a cache hit occurs. > > mem_cache_nums_ records the current number of KV entries stored in memory....

[Usage]: mem_cache_hit_nums and mem_cache_nums metrics not exposed via /metrics endpoint

> > > 在验证过程中，我发现了一个问题： > > > mem_cache_hit_nums_ 每次缓存命中时递增。 > > > mem_cache_nums_ 记录内存中存储的当前 KV 条目数。 > > > 分析 > > > 在 calculate_cache_stats() 方法中，缓存命中率的计算公式如下： > > > mem_cache_hit_nums_...