ronaldpanape

Results 2 comments of


                                            ronaldpanape

Recent vLLMs ask for too much memory: ValueError: No available memory for the cache blocks. Try increasing `gpu_memory_utilization` when initializing the engine.

having the same issue on cuda 11.8 and vllm 0.2.5 and 0.2.6

Roadmap to GA

when is the estimated GA?