zj

I deployed this repository on my dev server, and started `app.py`, but I noticed that the process occupys much system RAM (~80GB) after processing 7 or 8 images, and the...

@jklj077 大佬您好，我看您在 #576 中用padded的方式解决了qwen2 72b等模型awq量化后，与vllm kennel数不匹配的问题。您当时解决问题的代码如下： `import torch from torch.nn import functional as F from transformers import AutoModelForCausalLM # must use AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2-72B-Instruct", torch_dtype="auto") # this size is...