Does OpenGVLab/VisualPRM-8B support vllm inference?

[waltonfuture]

thanks for your great work. I hope to use vllm to speed up the VisualPRM-8B. Does it support vllm?

[Weiyun1025]

Thank you for your interest in our work. The model architecture of VisualPRM is the same with InternVL, so this model supports vllm inference. However, during the evaluation stage of VisualPRM, we use + as the placeholder and obtain the step score by one forward pass. Therefore, the inference cost of vllm and huggingface should be comparable. You can refer to our code for more details.