Billy Cao
Yes i meant the gptq hf branch. I figured it out myself by removing all quip and marlin codes and it works for me.
我有一样的问题，而且推理出的结果完全不对 CUDA 11.8, CUDNN 8.9, torch 2.1.0.dev20230425+cu118  
It is because of lack of AVX512, it has already been confirmed by https://github.com/PaddlePaddle/PaddleOCR/issues/10346, https://github.com/PaddlePaddle/PaddleOCR/issues/10675. I reproduce on both a Xeon and a AMD Ryzen 3, both don't support AVX512)....
Well as I said it is not a intel issue as i have it on AMD too. But if by "intel" you mean intel x86 then sure.
> Considering this works on paddlepaddle 2.4.2 BTW this dont work on 2.4.2, it just defaults to some alternate, unusablely slow code path, which is https://github.com/PaddlePaddle/PaddleOCR/issues/10346 So something changed from...
Thanks for the fix
我就按照教程的配置，最新的3.0beta都还可以用
@NielsRogge I thought the pad token id being wrong was fixed as mentioned by you in https://huggingface.co/llava-hf/llava-v1.6-mistral-7b-hf/discussions/2
So this would close https://github.com/huggingface/transformers/issues/29832 right? Any idea on if this change could solve https://github.com/huggingface/transformers/issues/29835?
Just got the gradio web demo to work, see linked PR.