micronetboy comments

Results 7 comments of


                                            micronetboy

Recent vLLMs ask for too much memory: ValueError: No available memory for the cache blocks. Try increasing `gpu_memory_utilization` when initializing the engine.

same here

Recent vLLMs ask for too much memory: ValueError: No available memory for the cache blocks. Try increasing `gpu_memory_utilization` when initializing the engine.

same here

[Feature]: Can it support the API for docking openai related models

汉奸版主小动作咋这么多呢删提议，不回答问题强制关闭问题，要求用英文提问! 跪着舔久了没膝盖了是吗?

[BUG] A100 80G，vLLM 运行 qwen-7b-chat 报错 torch.cuda.OutOfMemoryError。实际可用显存有 70G

是的，占用了 18G。还剩余 60 G，也应该能跑起来吧？

💡 [Question] - 微调数据格式，from字段可以是回答问题的人物吗？

您好，你的数据格式是什么样的？

💡 [Question] - QwenCPP Python Binding 如何支持 BLAS CPU 加速

python binding 如何设置 OpenBLAS 在CPU上加速？谢谢 python binding 如何设置 cuBLAS 在GPU上加速？谢谢

💡 [Question] - QwenCPP Python Binding 如何支持 BLAS CPU 加速

llama-cpp-python 有如下的选项，qwen-cpp 是否有类似的选项？ On Linux and Mac you set the CMAKE_ARGS like this: CMAKE_ARGS="-DLLAMA_BLAS=ON -DLLAMA_BLAS_VENDOR=OpenBLAS" pip install llama-cpp-python

micronetboy

Recent vLLMs ask for too much memory: ValueError: No available memory for the cache blocks. Try increasing `gpu_memory_utilization` when initializing the engine.

Recent vLLMs ask for too much memory: ValueError: No available memory for the cache blocks. Try increasing `gpu_memory_utilization` when initializing the engine.

[Feature]: Can it support the API for docking openai related models

[BUG] A100 80G，vLLM 运行 qwen-7b-chat 报错 torch.cuda.OutOfMemoryError。实际可用显存有 70G

💡 [Question] - 微调数据格式，from字段可以是回答问题的人物吗？

💡 [Question] - QwenCPP Python Binding 如何 支持 BLAS CPU 加速

💡 [Question] - QwenCPP Python Binding 如何 支持 BLAS CPU 加速

💡 [Question] - QwenCPP Python Binding 如何支持 BLAS CPU 加速

💡 [Question] - QwenCPP Python Binding 如何支持 BLAS CPU 加速