[BUG] 微调时数据集中图片无法正确加载

[Ataraxy33]

是否已有关于该错误的issue或讨论？ | Is there an existing issue / discussion for this?

• [X] 我已经搜索过已有的issues和讨论 | I have searched the existing issues / discussions

该问题是否在FAQ中有解答？ | Is there an existing answer for this in FAQ?

• [X] 我已经搜索过FAQ | I have searched FAQ

当前行为 | Current Behavior

当使用脚本LoRa微调Qwen-VL模型时，如果加载的data中有图片路径时，在.cache缓存文件中则会报错如下： ······ File "/home/zm2024/.cache/huggingface/modules/transformers_modules/Qwen-VL-Chat/modeling_qwen.py", line 657, in forward hidden_states[i][a + 1 : b] = images[idx] RuntimeError: a view of a leaf Variable that requires grad is being used in an in-place operation.

如果加载的数据集中没有图片，仅有文本输入，则可以正确开始训练。

期望行为 | Expected Behavior

No response

复现方法 | Steps To Reproduce

No response

运行环境 | Environment

- OS: Ubuntu 20.04
- Python: 3.10
- Transformers: 4.32.0
- PyTorch: 2.2.1
- CUDA (`python -c 'import torch; print(torch.version.cuda)'`):12.1

备注 | Anything else?

No response

[J0eky]

hi, have you solved it? @Ataraxy33

[Ataraxy33]

hi, have you solved it? @Ataraxy33

yes, I use another tool to finetune it and it works. Please check this link: https://github.com/modelscope/swift

[1180300419]

@J0eky Hey，have you solved it？

[miovovo]

@Ataraxy33 hello! Your link here seems to be invalid, would you mind to share again? I got the same bug

[Ataraxy33]

@Ataraxy33 hello! Your link here seems to be invalid, would you mind to share again? I got the same bug

Okay, please check this new link: https://github.com/modelscope/swift

[wade30822]

请问下训练集的图片大小和数据量保持在多少比较合适