Teng Ma
Teng Ma
TODO: 1. Reduce serialization overhead 2. Should replace zmq with a higher-performance notification mechanism. 3. one-to-many send, like broadcast.
@staryxchen JFYI, Concurrent register memory is an important optimization. CC. @alogfans
@zhongzhouTan-coder
Could you add this feature to #414 ? @xiaguan
/gemini review it
> ``` > constexpr auto kInitialBackoffMs = std::chrono::milliseconds(1); > constexpr auto kMaxBackoffMs = std::chrono::milliseconds(100); > ``` > > Would using milliseconds be too high for RDMA transmission? > > In...
> > Sure. microseconds should be better. QQ: for 2*400Gbps environements, there are NV connectX NIC or other types? > > four NV connectX NICs, two as preferred and the...
> Great! BTW, is cache-aware scheduler in the roadmap? Good idea! Is the Cache-aware scheduler on the master side or another implementation?
> > * KVCache Reuse between prefill nodes > > @stmatengss Hi, I wanna try it, thanks~ Cool! Hope to see your PR soon!
Actually, Mooncake doesn't fully support IPv6. Some components such as Transport can support IPv6, but not for http metadata server and coro rpc.