Lu Changqi
Lu Changqi
hello, have you solved it?
> > 您可以改用VOC_CLASSES = [（ > > 'feng'）] > > it doesnt work 您好！我遇到了同样的问题，您解决了吗？可以探讨一下！
> Excuse me, have you solved this problem? hello，have you solved the problem?Thank you!
@ApostaC When using --enable-prefix-caching and encountering a cache hit, the same error occurs. However, with --enable-prefix-caching enabled, this error can be avoided, and the TTFT is reduced during cache hits....
In this PR https://github.com/vllm-project/vllm/pull/13747#issuecomment-2687039690, I mentioned this bug. cc @LucasWilkinson
@simon-mo hi，regarding the topic “KV cache offload to CPU and disk”, I previously implemented a version that stores kv cache in a local file(https://github.com/vllm-project/vllm/pull/8018). Of course, I also did relevant...
Hi! I have an idea. Can we support a key-value database similar to valkey (Redis over RDMA)? Among them, the key is the hash value of the token. In this...
Ping! Could you give me some advice?
> pin_memory has a great impact on swapping blocks. > > more specifically > > benchmarks/kernels/benchmark_swap_blocks.py > > ``` > + from light_vllm.utils import is_pin_memory_available > + pin_memory = is_pin_memory_available()...