InternLM-XComposer icon indicating copy to clipboard operation
InternLM-XComposer copied to clipboard

Fix a bug in the huggingface model

Open YushunXiang opened this issue 1 year ago • 1 comments

When I was fine-tuning the model, the bug happened. The file finetune/data_mix.py, line 137, in get_item. The dictionary sample has the key image. But in the huggingface model, for example, the file internlm-xcomposer-vl-7b/modeling_InternLM_XComposer.py, line 381, in forward. has_img = 'images' in samples.keys() will never have the key named images. So we should change it to has_img = 'image' in samples.keys(). image image

After change: image

YushunXiang avatar Jan 18 '24 14:01 YushunXiang

Actually, I think chage has_img = 'images' in samples.keys() to has_img = samples['data_type'] == 'multi' is better and more elegant.

YushunXiang avatar Jan 18 '24 14:01 YushunXiang