volcengine
KeyError: 'visual.patch_embed.proj.weight'
我用我自己的模型路径和数据运行bash examples/grpo_trainer/run_qwen2_5_vl-7b.sh 。一直出现File "/usr/local/lib/python3.11/site-packages/vllm/model_executor/models/qwen2.py", line 405, in load_weights param = params_dict[name] ~~~~~~~~~~~^^^^^^ KeyError: 'visual.patch_embed.proj.weight' 这个问题，这是怎么回事呀
You can downgrade the transformers to 4.51.0 to solve the problem temporarily. There are some conflicts here.
同错误，请问你的vllm是多少，我的vllm是0.8.5.post。安装此版本的transformers会出现“ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. vllm 0.8.5.post1 requires transformers>=4.51.1,”
同错误，请问你的vllm是多少，我的vllm是0.8.5.post。安装此版本的transformers会出现“ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. vllm 0.8.5.post1 requires transformers>=4.51.1,”
I have tried the transformers==4.51.1 with vllm==0.8.5.post1. Right now, it works.
同错误，请问你的vllm是多少，我的vllm是0.8.5.post。安装此版本的transformers会出现“ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. vllm 0.8.5.post1 requires transformers>=4.51.1,”
You can try to ignore the error, or you can try to use vllm==0.8.2.
Does anyone have insight into why transformers >=4.52 throws this error? I'm working with a finetuned 3B Qwen model that is only compatible with transformers 4.52 and higher, so it is not an option to use transformers 4.51.0
Does anyone have insight into why transformers >=4.52 throws this error? I'm working with a finetuned 3B Qwen model that is only compatible with transformers 4.52 and higher, so it is not an option to use transformers 4.51.0
I think you can try to modify the key's name in the model weights or in the code to make sure that the framework can correctly load the weight.
Does anyone have insight into why transformers >=4.52 throws this error? I'm working with a finetuned 3B Qwen model that is only compatible with transformers 4.52 and higher, so it is not an option to use transformers 4.51.0
You can see 19054 at vllm to check the results, it is the model weight conversion change in the transformers.
So, does version 0.4.0 of verl solve this issue? I'm using transformers==4.52.4 and vllm==0.8.5, and I’m still getting similar error messages.
If it's not yet patched you can call this function at the beginning of your python script
def patch_qwen_weights_vllm():
"""
Patch weight names of qwen multimodal models consistently with transformers==4.52
See https://github.com/vllm-project/vllm/pull/19054
"""
import vllm
from vllm.model_executor.models.utils import WeightsMapper
vllm.model_executor.models.ModelRegistry.models["Qwen2_5_VLForConditionalGeneration"].load_model_cls().hf_to_vllm_mapper = WeightsMapper(
orig_to_new_prefix={
# mapping for new names in checkpoint saved after transformers v4.52
"model.language_model.": "language_model.model.",
"model.visual.": "visual.",
# mapping for original checkpoint
"lm_head.": "language_model.lm_head.",
"model.": "language_model.model.",
}
)
vllm.model_executor.models.ModelRegistry.models["Qwen2VLForConditionalGeneration"].load_model_cls().hf_to_vllm_mapper = WeightsMapper(
orig_to_new_prefix={
# mapping for new names in checkpoint saved after transformers v4.52
"model.language_model.": "language_model.model.",
"model.visual.": "visual.",
# mapping for original checkpoint
"lm_head.": "language_model.lm_head.",
"model.": "language_model.model.",
}
)
print("### Patch to vllm qwen modelling applied successfully.")
