OpenBMB
[BUG] <title>使用llama-factory微调出现“operands could not be broadcast together with shapes (336,602,3) (3,3) ”
是否已有关于该错误的issue或讨论？ | Is there an existing issue / discussion for this?
- [x] 我已经搜索过已有的issues和讨论 | I have searched the existing issues / discussions
该问题是否在FAQ中有解答？ | Is there an existing answer for this in FAQ?
- [x] 我已经搜索过FAQ | I have searched FAQ
当前行为 | Current Behavior
No response
期望行为 | Expected Behavior
运行代码： CUDA_VISIBLE_DEVICES=0 llamafactory-cli train configs/minicpmo_2_6_lora_sft.yaml
报错：
[INFO|2025-02-23 16:59:12] llamafactory.data.template:157 >> Add <|im_end|> to stop words.
[INFO|2025-02-23 16:59:12] llamafactory.data.loader:157 >> Loading dataset demo_minicpm_llama_factory.json...
num_proc must be <= 10. Reducing num_proc to 10 for dataset of size 10.
Converting format of dataset (num_proc=10): 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:00<00:00, 42.83 examples/s]
num_proc must be <= 10. Reducing num_proc to 10 for dataset of size 10.
Running tokenizer on dataset (num_proc=10): 0%| | 0/10 [00:02<?, ? examples/s]
multiprocess.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/opt/miniconda3/envs/llamaFactory/lib/python3.10/site-packages/multiprocess/pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "/opt/miniconda3/envs/llamaFactory/lib/python3.10/site-packages/datasets/utils/py_utils.py", line 678, in _write_generator_to_queue
for i, result in enumerate(func(**kwargs)):
File "/opt/miniconda3/envs/llamaFactory/lib/python3.10/site-packages/datasets/arrow_dataset.py", line 3476, in _map_single
batch = apply_function_on_filtered_inputs(
File "/opt/miniconda3/envs/llamaFactory/lib/python3.10/site-packages/datasets/arrow_dataset.py", line 3338, in apply_function_on_filtered_inputs
processed_inputs = function(*fn_args, *additional_args, **fn_kwargs)
File "/opt/miniconda3/envs/llamaFactory/third_partys/LLaMA-Factory-main/src/llamafactory/data/processor/supervised.py", line 99, in preprocess_dataset
input_ids, labels = self._encode_data_example(
File "/opt/miniconda3/envs/llamaFactory/third_partys/LLaMA-Factory-main/src/llamafactory/data/processor/supervised.py", line 43, in _encode_data_example
messages = self.template.mm_plugin.process_messages(prompt + response, images, videos, audios, self.processor)
File "/opt/miniconda3/envs/llamaFactory/third_partys/LLaMA-Factory-main/src/llamafactory/data/mm_plugin.py", line 550, in process_messages
mm_inputs = self._get_mm_inputs(images, [], [], processor)
File "/opt/miniconda3/envs/llamaFactory/third_partys/LLaMA-Factory-main/src/llamafactory/data/mm_plugin.py", line 631, in _get_mm_inputs
image_inputs = image_processor(
File "/opt/miniconda3/envs/llamaFactory/lib/python3.10/site-packages/transformers/image_processing_utils.py", line 42, in call
return self.preprocess(images, **kwargs)
File "/home/ime/.cache/huggingface/modules/transformers_modules/MiniCPM-o-2_6/image_processing_minicpmv.py", line 381, in preprocess
image_patches = [
File "/home/ime/.cache/huggingface/modules/transformers_modules/MiniCPM-o-2_6/image_processing_minicpmv.py", line 382, in
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/miniconda3/envs/llamaFactory/bin/llamafactory-cli", line 8, in
复现方法 | Steps To Reproduce
No response
运行环境 | Environment
Package Version Editable project location
------------------------- ------------ -----------------------------------------------------------------
accelerate 1.2.1
aiofiles 23.2.1
aiohappyeyeballs 2.4.6
aiohttp 3.11.12
aiosignal 1.3.2
annotated-types 0.7.0
anyio 4.8.0
async-timeout 5.0.1
attrs 25.1.0
audioread 3.0.1
av 14.1.0
certifi 2022.12.7
cffi 1.17.1
charset-normalizer 2.1.1
click 8.1.8
contourpy 1.3.1
cycler 0.12.1
datasets 3.2.0
decorator 5.2.0
deepspeed 0.16.2
dill 0.3.8
docstring_parser 0.16
einops 0.8.1
einx 0.3.0
encodec 0.1.1
exceptiongroup 1.2.2
fastapi 0.115.8
ffmpy 0.5.0
filelock 3.17.0
fire 0.7.0
flash-attn 2.7.3
fonttools 4.56.0
frozendict 2.4.6
frozenlist 1.5.0
fsspec 2024.9.0
gradio 5.12.0
gradio_client 1.5.4
h11 0.14.0
hjson 3.1.0
httpcore 1.0.7
httpx 0.28.1
huggingface-hub 0.29.1
idna 3.4
jieba 0.42.1
Jinja2 3.1.5
joblib 1.4.2
jsonschema-specifications 2024.10.1
kiwisolver 1.4.8
lazy_loader 0.4
librosa 0.10.2.post1
llamafactory 0.9.2.dev0 /opt/miniconda3/envs/llamaFactory/third_partys/LLaMA-Factory-main
llvmlite 0.44.0
markdown-it-py 3.0.0
MarkupSafe 2.1.5
matplotlib 3.10.0
mdurl 0.1.2
mpmath 1.3.0
msgpack 1.1.0
multidict 6.1.0
multiprocess 0.70.16
networkx 3.4.2
ninja 1.11.1.3
nltk 3.9.1
numba 0.61.0
numpy 1.26.4
nvidia-cublas-cu11 11.11.3.6
nvidia-cuda-cupti-cu11 11.8.87
nvidia-cuda-nvrtc-cu11 11.8.89
nvidia-cuda-runtime-cu11 11.8.89
nvidia-cudnn-cu11 8.7.0.84
nvidia-cufft-cu11 10.9.0.58
nvidia-curand-cu11 10.3.0.86
nvidia-cusolver-cu11 11.4.1.48
nvidia-cusparse-cu11 11.7.5.86
nvidia-ml-py 12.570.86
nvidia-nccl-cu11 2.19.3
nvidia-nvtx-cu11 11.8.86
orjson 3.10.15
packaging 24.2
pandas 2.2.3
peft 0.12.0
pillow 11.0.0
pip 25.0
platformdirs 4.3.6
pooch 1.8.2
propcache 0.3.0
protobuf 5.29.3
psutil 7.0.0
py-cpuinfo 9.0.0
pyarrow 19.0.1
pycparser 2.22
pydantic 2.10.6
pydantic_core 2.27.2
pydub 0.25.1
Pygments 2.19.1
pyparsing 3.2.1
python-dateutil 2.9.0.post0
python-multipart 0.0.20
pytz 2025.1
PyYAML 6.0.2
referencing 0.36.2
regex 2024.11.6
requests 2.32.3
rich 13.9.4
rouge-chinese 1.0.3
rpds-py 0.23.1
ruff 0.9.7
safehttpx 0.1.6
safetensors 0.5.2
scikit-learn 1.6.1
scipy 1.15.2
semantic-version 2.10.0
sentencepiece 0.2.0
setuptools 75.8.0
shellingham 1.5.4
shtab 1.7.1
six 1.17.0
sniffio 1.3.1
soundfile 0.13.1
soxr 0.5.0.post1
sse-starlette 2.2.1
starlette 0.45.3
sympy 1.13.3
termcolor 2.5.0
threadpoolctl 3.5.0
tiktoken 0.9.0
tokenizers 0.21.0
tomlkit 0.13.2
torch 2.2.0+cu118
torchaudio 2.2.0+cu118
torchvision 0.17.0+cu118
tqdm 4.67.1
transformers 4.49.0
triton 2.2.0
trl 0.9.6
typer 0.15.1
typing_extensions 4.12.2
tyro 0.8.14
tzdata 2025.1
urllib3 1.26.13
uvicorn 0.34.0
vector-quantize-pytorch 1.21.8
vocos 0.1.0
websockets 14.2
wheel 0.45.1
xxhash 3.5.0
yarl 1.18.3
备注 | Anything else?
No response
@li784 我重新安装了这个版本的transforms，这个问题跳过了 暂时推荐使用transformers==4.45.0，可以稳定跑通微调和推理
【重要】使用以下方式安装最新的llamafactory以及相应的库
git clone --depth 1 https://github.com/hiyouga/LLaMA-Factory.git
cd LLaMA-Factory
pip install -e ".[torch,metrics,deepspeed,minicpm_v]"
pip3 install transformers==4.45.0
pip3 install huggingface_hub==0.25.0
@li784 我重新安装了这个版本的transforms，这个问题跳过了 暂时推荐使用transformers==4.45.0，可以稳定跑通微调和推理
【重要】使用以下方式安装最新的llamafactory以及相应的库
git clone --depth 1 https://github.com/hiyouga/LLaMA-Factory.gitcd LLaMA-Factorypip install -e ".[torch,metrics,deepspeed,minicpm_v]"pip3 install transformers==4.45.0pip3 install huggingface_hub==0.25.0更新完transformers 报新的错误：
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train configs/minicpmo_2_6_lora_sft.yaml
Traceback (most recent call last):
File "/root/anaconda3/envs/llamafactory/bin/llamafactory-cli", line 5, in
Getting the same error as @everwind , is there any solution?
ImportError: cannot import name 'AutoModelForImageTextToText' from 'transformers' solved by install llama-factory tag 0.9.2
but it may occur another error [rank3]: Traceback (most recent call last):
[rank3]: File "/mnt/bella/users/zhaoshuaijiang/codes/LLaMA-Factory/src/llamafactory/launcher.py", line 23, in
@li784 我重新安装了这个版本的transforms，这个问题跳过了 暂时推荐使用transformers==4.45.0，可以稳定跑通微调和推理 【重要】使用以下方式安装最新的llamafactory以及相应的库
git clone --depth 1 https://github.com/hiyouga/LLaMA-Factory.gitcd LLaMA-Factorypip install -e ".[torch,metrics,deepspeed,minicpm_v]"pip3 install transformers==4.45.0pip3 install huggingface_hub==0.25.0更新完transformers 报新的错误：CUDA_VISIBLE_DEVICES=0 llamafactory-cli train configs/minicpmo_2_6_lora_sft.yaml Traceback (most recent call last): File "/root/anaconda3/envs/llamafactory/bin/llamafactory-cli", line 5, in from llamafactory.cli import main File "/root/paddlejob/workspace2/LLaMA-Factory/src/llamafactory/cli.py", line 21, in from . import launcher File "/root/paddlejob/workspace2/LLaMA-Factory/src/llamafactory/launcher.py", line 15, in from llamafactory.train.tuner import run_exp # use absolute import File "/root/paddlejob/workspace2/LLaMA-Factory/src/llamafactory/train/tuner.py", line 29, in from ..model import load_model, load_tokenizer File "/root/paddlejob/workspace2/LLaMA-Factory/src/llamafactory/model/init.py", line 15, in from .loader import load_config, load_model, load_tokenizer File "/root/paddlejob/workspace2/LLaMA-Factory/src/llamafactory/model/loader.py", line 19, in from transformers import ( ImportError: cannot import name 'AutoModelForImageTextToText' from 'transformers' (/root/anaconda3/envs/llamafactory/lib/python3.10/site-packages/transformers/init.py)
I changed the version of transformers to 4.48.3 and it worked.
