DaBossCoda

Results 4 comments of


                                            DaBossCoda

Recent vLLMs ask for too much memory: ValueError: No available memory for the cache blocks. Try increasing `gpu_memory_utilization` when initializing the engine.

same here

Feature request：support ExLlama

Also looking forward to exllamav2 support

memory leak

Also, having a memory leak issue. Can't seem to fix it :/

[Usage]: Flash Attention not working any more

Same