ms-swift
ms-swift copied to clipboard
Published
1 week ago
•
modelscope
modelscope
unsloth error when sft qwen2.5-vl-7b-instruct
Describe the bug
Traceback (most recent call last):
File "/root/miniconda3/envs/py310/lib/python3.10/site-packages/swift/cli/sft.py", line 5, in <module>
sft_main()
File "/root/miniconda3/envs/py310/lib/python3.10/site-packages/swift/llm/train/sft.py", line 257, in sft_main
return SwiftSft(args).main()
File "/root/miniconda3/envs/py310/lib/python3.10/site-packages/swift/llm/train/sft.py", line 30, in __init__
self._prepare_model_tokenizer()
File "/root/miniconda3/envs/py310/lib/python3.10/site-packages/swift/llm/train/sft.py", line 62, in _prepare_model_tokenizer
self.model, self.processor = args.get_model_processor()
File "/root/miniconda3/envs/py310/lib/python3.10/site-packages/swift/llm/argument/base_args/base_args.py", line 271, in get_model_processor
return load_by_unsloth(self)
File "/root/miniconda3/envs/py310/lib/python3.10/site-packages/swift/llm/model/register.py", line 142, in load_by_unsloth
model, processor = UnslothModel.from_pretrained(
File "/root/miniconda3/envs/py310/lib/python3.10/site-packages/unsloth/models/loader.py", line 666, in from_pretrained
model_types, supports_sdpa = unsloth_compile_transformers(
TypeError: cannot unpack non-iterable NoneType object
Your hardware and system info ubuntu 24 pytorch 2.6 4090 GPU CUDA 12.4
Package Version Editable project location
----------------------------- ------------ ------------------------------------
absl-py 2.1.0
accelerate 1.1.1
addict 2.4.0
aiofiles 23.2.1
aiohappyeyeballs 2.4.3
aiohttp 3.11.2
aiosignal 1.3.1
aliyun-python-sdk-core 2.16.0
aliyun-python-sdk-kms 2.16.5
altair 5.4.1
annotated-types 0.7.0
anyio 4.6.2.post1
asttokens 2.4.1
async-timeout 5.0.1
attrdict 2.0.1
attributedict 0.3.0
attrs 24.2.0
auto_gptq 0.7.1
autoawq 0.2.7.post1
av 13.1.0
awq 0.1.0 /root/autodl-tmp/clean_jsonl/llm-awq
binpacking 1.5.2
bitsandbytes 0.45.4
blessings 1.7
cachetools 5.5.0
certifi 2024.8.30
cffi 1.17.1
chardet 5.2.0
charset-normalizer 3.4.0
click 8.1.7
codecov 2.1.13
colorama 0.4.6
coloredlogs 15.0.1
colour-runner 0.1.1
comm 0.2.2
contourpy 1.3.1
coverage 7.6.7
cpm-kernels 1.0.11
crcmod 1.7
cryptography 43.0.3
cut-cross-entropy 25.1.1
cycler 0.12.1
dacite 1.8.1
DataProperty 1.0.1
datasets 3.2.0
debugpy 1.8.8
decorator 5.1.1
decord 0.6.0
deepdiff 8.0.1
diffusers 0.32.2
dill 0.3.8
distlib 0.3.9
distro 1.9.0
docstring_parser 0.16
einops 0.8.0
exceptiongroup 1.2.2
executing 2.1.0
fastapi 0.115.5
ffmpy 0.4.0
filelock 3.16.1
flash-attn 2.7.0.post2
fonttools 4.55.0
frozenlist 1.5.0
fsspec 2024.2.0
future 1.0.0
gekko 1.2.1
gradio 5.18.0
gradio_client 1.7.2
grpcio 1.68.0
h11 0.14.0
hf_transfer 0.1.9
httpcore 1.0.7
httpx 0.27.2
huggingface-hub 0.29.1
humanfriendly 10.0
idna 3.10
importlib_metadata 8.5.0
inspecta 0.1.3
ipykernel 6.29.5
ipython 8.29.0
jedi 0.19.2
jieba 0.42.1
Jinja2 3.1.3
jiter 0.7.1
jlab_html_extension 0.1.0
jmespath 0.10.0
joblib 1.4.2
jsonlines 4.0.0
jsonschema 4.23.0
jsonschema-specifications 2024.10.1
jupyter_client 8.6.3
jupyter_core 5.7.2
kiwisolver 1.4.7
linkify-it-py 2.0.3
lm-eval 0.3.0
Markdown 3.7
markdown-it-py 2.2.0
MarkupSafe 2.1.5
matplotlib 3.9.2
matplotlib-inline 0.1.7
mbstrdecoder 1.1.3
mdit-py-plugins 0.3.3
mdurl 0.1.2
modelscope 1.23.1
mpmath 1.3.0
ms_swift 3.1.1.post1
multidict 6.1.0
multiprocess 0.70.16
narwhals 1.14.0
nest-asyncio 1.6.0
networkx 3.2.1
nltk 3.9.1
numexpr 2.10.1
numpy 1.26.3
nvidia-cublas-cu12 12.4.5.8
nvidia-cuda-cupti-cu12 12.4.127
nvidia-cuda-nvrtc-cu12 12.4.127
nvidia-cuda-runtime-cu12 12.4.127
nvidia-cudnn-cu12 9.1.0.70
nvidia-cufft-cu12 11.2.1.3
nvidia-curand-cu12 10.3.5.147
nvidia-cusolver-cu12 11.6.1.9
nvidia-cusparse-cu12 12.3.1.170
nvidia-cusparselt-cu12 0.6.2
nvidia-nccl-cu12 2.21.5
nvidia-nvjitlink-cu12 12.4.127
nvidia-nvtx-cu12 12.4.127
openai 1.54.4
opencv-python 4.11.0.86
optimum 1.23.3
orderly-set 5.2.2
orjson 3.10.11
oss2 2.19.1
packaging 24.2
pandas 2.2.3
parso 0.8.4
pathvalidate 3.2.1
peft 0.12.0
pexpect 4.9.0
pillow 10.2.0
pip 24.2
platformdirs 4.3.6
pluggy 1.5.0
portalocker 3.0.0
prompt_toolkit 3.0.48
propcache 0.2.0
protobuf 3.20.3
psutil 6.1.0
ptyprocess 0.7.0
pure_eval 0.2.3
pyarrow 18.0.0
pyav 13.1.0
pybind11 2.13.6
pycountry 24.6.1
pycparser 2.22
pycryptodome 3.21.0
pydantic 2.9.2
pydantic_core 2.23.4
pydub 0.25.1
Pygments 2.18.0
pyparsing 3.2.0
pyproject-api 1.8.0
pytablewriter 1.2.0
python-dateutil 2.9.0.post0
python-multipart 0.0.20
pytz 2024.2
PyYAML 6.0.2
pyzmq 26.2.0
qwen-vl-utils 0.0.8
referencing 0.35.1
regex 2024.11.6
requests 2.32.3
rich 13.9.4
rootpath 0.1.1
rouge 1.0.1
rouge_score 0.1.2
rpds-py 0.21.0
ruff 0.9.7
sacrebleu 1.5.0
safehttpx 0.1.6
safetensors 0.4.5
scikit-learn 1.5.2
scipy 1.14.1
semantic-version 2.10.0
sentencepiece 0.2.0
setuptools 69.5.1
shellingham 1.5.4
shtab 1.7.1
simplejson 3.19.3
six 1.16.0
sniffio 1.3.1
sortedcontainers 2.4.0
sqlitedict 2.1.0
stack-data 0.6.3
starlette 0.41.2
sympy 1.13.1
tabledata 1.3.3
tcolorpy 0.1.6
tensorboard 2.18.0
tensorboard-data-server 0.7.2
termcolor 2.5.0
texttable 1.7.0
threadpoolctl 3.5.0
tiktoken 0.8.0
tokenizers 0.21.0
toml 0.10.2
tomli 2.1.0
tomlkit 0.12.0
torch 2.6.0+cu124
torchaudio 2.6.0+cu124
torchvision 0.21.0+cu124
tornado 6.4.1
tox 4.23.2
tqdm 4.67.0
tqdm-multiprocess 0.0.11
traitlets 5.14.3
transformers 4.50.0.dev0
transformers-stream-generator 0.0.5
triton 3.2.0
trl 0.15.2
typepy 1.3.2
typer 0.13.0
typing_extensions 4.12.2
tyro 0.8.14
tzdata 2024.2
uc-micro-py 1.0.3
unsloth 2025.3.19
unsloth_zoo 2025.3.17
urllib3 2.2.3
uvicorn 0.32.0
virtualenv 20.27.1
wcwidth 0.2.13
websockets 12.0
Werkzeug 3.1.3
wheel 0.44.0
xformers 0.0.29.post3
xxhash 3.5.0
yarl 1.17.1
zipp 3.21.0
zstandard 0.23.0
Additional context 使用 unsloth 微调 qwen2.5-vl-7B-instruct 时报错。启动命令：
FPS_MAX_FRAMES=1 FPS=2 CUDA_VISIBLE_DEVICES=0 SIZE_FACTOR=2 MAX_PIXELS=200704 swift sft \
--model_type qwen2_5_vl \
--model Qwen2.5-VL-7B-Instruct \
--tuner_backend unsloth \
--quant_bits 4 \
--num_train_epochs 2 \
--learning_rate 1e-5 \
--lora_rank 128 \
--lora_alpha 64 \
--per_device_train_batch_size 1 \
--per_device_eval_batch_size 1 \
--train_type lora \
--target_modules all-linear \
--max_length 350 \
--gradient_accumulation_steps 16 \
--eval_steps 1000000 \
--save_steps 100 \
--freeze_vit false \
--freeze_llm false \
--split_dataset_ratio 0.001 \
--torch_dtype bfloat16 \
--save_total_limit 2 \
--data_seed 299 \
--dataset clean_jsonl/clean/data.jsonl
I didn't meet this problem with this command:
FPS_MAX_FRAMES=1 FPS=2 CUDA_VISIBLE_DEVICES=0 SIZE_FACTOR=2 MAX_PIXELS=200704 swift sft \
--model Qwen/Qwen2.5-VL-7B-Instruct \
--tuner_backend unsloth \
--quant_bits 4 \
--num_train_epochs 2 \
--learning_rate 1e-5 \
--lora_rank 128 \
--lora_alpha 64 \
--per_device_train_batch_size 1 \
--per_device_eval_batch_size 1 \
--train_type lora \
--target_modules q_proj v_proj k_proj \
--max_length 350 \
--gradient_accumulation_steps 16 \
--eval_steps 1000000 \
--save_steps 100 \
--freeze_vit false \
--freeze_llm false \
--split_dataset_ratio 0.001 \
--torch_dtype bfloat16 \
--save_total_limit 2 \
--data_seed 299 \
--dataset swift/ScienceQA#1000
Can you try to upgrade unsloth/transformers/trl/accelerate to the latest version?
Describe the bug
Traceback (most recent call last): File "/root/miniconda3/envs/py310/lib/python3.10/site-packages/swift/cli/sft.py", line 5, in <module> sft_main() File "/root/miniconda3/envs/py310/lib/python3.10/site-packages/swift/llm/train/sft.py", line 257, in sft_main return SwiftSft(args).main() File "/root/miniconda3/envs/py310/lib/python3.10/site-packages/swift/llm/train/sft.py", line 30, in __init__ self._prepare_model_tokenizer() File "/root/miniconda3/envs/py310/lib/python3.10/site-packages/swift/llm/train/sft.py", line 62, in _prepare_model_tokenizer self.model, self.processor = args.get_model_processor() File "/root/miniconda3/envs/py310/lib/python3.10/site-packages/swift/llm/argument/base_args/base_args.py", line 271, in get_model_processor return load_by_unsloth(self) File "/root/miniconda3/envs/py310/lib/python3.10/site-packages/swift/llm/model/register.py", line 142, in load_by_unsloth model, processor = UnslothModel.from_pretrained( File "/root/miniconda3/envs/py310/lib/python3.10/site-packages/unsloth/models/loader.py", line 666, in from_pretrained model_types, supports_sdpa = unsloth_compile_transformers( TypeError: cannot unpack non-iterable NoneType objectYour hardware and system info ubuntu 24 pytorch 2.6 4090 GPU CUDA 12.4
Package Version Editable project location ----------------------------- ------------ ------------------------------------ absl-py 2.1.0 accelerate 1.1.1 addict 2.4.0 aiofiles 23.2.1 aiohappyeyeballs 2.4.3 aiohttp 3.11.2 aiosignal 1.3.1 aliyun-python-sdk-core 2.16.0 aliyun-python-sdk-kms 2.16.5 altair 5.4.1 annotated-types 0.7.0 anyio 4.6.2.post1 asttokens 2.4.1 async-timeout 5.0.1 attrdict 2.0.1 attributedict 0.3.0 attrs 24.2.0 auto_gptq 0.7.1 autoawq 0.2.7.post1 av 13.1.0 awq 0.1.0 /root/autodl-tmp/clean_jsonl/llm-awq binpacking 1.5.2 bitsandbytes 0.45.4 blessings 1.7 cachetools 5.5.0 certifi 2024.8.30 cffi 1.17.1 chardet 5.2.0 charset-normalizer 3.4.0 click 8.1.7 codecov 2.1.13 colorama 0.4.6 coloredlogs 15.0.1 colour-runner 0.1.1 comm 0.2.2 contourpy 1.3.1 coverage 7.6.7 cpm-kernels 1.0.11 crcmod 1.7 cryptography 43.0.3 cut-cross-entropy 25.1.1 cycler 0.12.1 dacite 1.8.1 DataProperty 1.0.1 datasets 3.2.0 debugpy 1.8.8 decorator 5.1.1 decord 0.6.0 deepdiff 8.0.1 diffusers 0.32.2 dill 0.3.8 distlib 0.3.9 distro 1.9.0 docstring_parser 0.16 einops 0.8.0 exceptiongroup 1.2.2 executing 2.1.0 fastapi 0.115.5 ffmpy 0.4.0 filelock 3.16.1 flash-attn 2.7.0.post2 fonttools 4.55.0 frozenlist 1.5.0 fsspec 2024.2.0 future 1.0.0 gekko 1.2.1 gradio 5.18.0 gradio_client 1.7.2 grpcio 1.68.0 h11 0.14.0 hf_transfer 0.1.9 httpcore 1.0.7 httpx 0.27.2 huggingface-hub 0.29.1 humanfriendly 10.0 idna 3.10 importlib_metadata 8.5.0 inspecta 0.1.3 ipykernel 6.29.5 ipython 8.29.0 jedi 0.19.2 jieba 0.42.1 Jinja2 3.1.3 jiter 0.7.1 jlab_html_extension 0.1.0 jmespath 0.10.0 joblib 1.4.2 jsonlines 4.0.0 jsonschema 4.23.0 jsonschema-specifications 2024.10.1 jupyter_client 8.6.3 jupyter_core 5.7.2 kiwisolver 1.4.7 linkify-it-py 2.0.3 lm-eval 0.3.0 Markdown 3.7 markdown-it-py 2.2.0 MarkupSafe 2.1.5 matplotlib 3.9.2 matplotlib-inline 0.1.7 mbstrdecoder 1.1.3 mdit-py-plugins 0.3.3 mdurl 0.1.2 modelscope 1.23.1 mpmath 1.3.0 ms_swift 3.1.1.post1 multidict 6.1.0 multiprocess 0.70.16 narwhals 1.14.0 nest-asyncio 1.6.0 networkx 3.2.1 nltk 3.9.1 numexpr 2.10.1 numpy 1.26.3 nvidia-cublas-cu12 12.4.5.8 nvidia-cuda-cupti-cu12 12.4.127 nvidia-cuda-nvrtc-cu12 12.4.127 nvidia-cuda-runtime-cu12 12.4.127 nvidia-cudnn-cu12 9.1.0.70 nvidia-cufft-cu12 11.2.1.3 nvidia-curand-cu12 10.3.5.147 nvidia-cusolver-cu12 11.6.1.9 nvidia-cusparse-cu12 12.3.1.170 nvidia-cusparselt-cu12 0.6.2 nvidia-nccl-cu12 2.21.5 nvidia-nvjitlink-cu12 12.4.127 nvidia-nvtx-cu12 12.4.127 openai 1.54.4 opencv-python 4.11.0.86 optimum 1.23.3 orderly-set 5.2.2 orjson 3.10.11 oss2 2.19.1 packaging 24.2 pandas 2.2.3 parso 0.8.4 pathvalidate 3.2.1 peft 0.12.0 pexpect 4.9.0 pillow 10.2.0 pip 24.2 platformdirs 4.3.6 pluggy 1.5.0 portalocker 3.0.0 prompt_toolkit 3.0.48 propcache 0.2.0 protobuf 3.20.3 psutil 6.1.0 ptyprocess 0.7.0 pure_eval 0.2.3 pyarrow 18.0.0 pyav 13.1.0 pybind11 2.13.6 pycountry 24.6.1 pycparser 2.22 pycryptodome 3.21.0 pydantic 2.9.2 pydantic_core 2.23.4 pydub 0.25.1 Pygments 2.18.0 pyparsing 3.2.0 pyproject-api 1.8.0 pytablewriter 1.2.0 python-dateutil 2.9.0.post0 python-multipart 0.0.20 pytz 2024.2 PyYAML 6.0.2 pyzmq 26.2.0 qwen-vl-utils 0.0.8 referencing 0.35.1 regex 2024.11.6 requests 2.32.3 rich 13.9.4 rootpath 0.1.1 rouge 1.0.1 rouge_score 0.1.2 rpds-py 0.21.0 ruff 0.9.7 sacrebleu 1.5.0 safehttpx 0.1.6 safetensors 0.4.5 scikit-learn 1.5.2 scipy 1.14.1 semantic-version 2.10.0 sentencepiece 0.2.0 setuptools 69.5.1 shellingham 1.5.4 shtab 1.7.1 simplejson 3.19.3 six 1.16.0 sniffio 1.3.1 sortedcontainers 2.4.0 sqlitedict 2.1.0 stack-data 0.6.3 starlette 0.41.2 sympy 1.13.1 tabledata 1.3.3 tcolorpy 0.1.6 tensorboard 2.18.0 tensorboard-data-server 0.7.2 termcolor 2.5.0 texttable 1.7.0 threadpoolctl 3.5.0 tiktoken 0.8.0 tokenizers 0.21.0 toml 0.10.2 tomli 2.1.0 tomlkit 0.12.0 torch 2.6.0+cu124 torchaudio 2.6.0+cu124 torchvision 0.21.0+cu124 tornado 6.4.1 tox 4.23.2 tqdm 4.67.0 tqdm-multiprocess 0.0.11 traitlets 5.14.3 transformers 4.50.0.dev0 transformers-stream-generator 0.0.5 triton 3.2.0 trl 0.15.2 typepy 1.3.2 typer 0.13.0 typing_extensions 4.12.2 tyro 0.8.14 tzdata 2024.2 uc-micro-py 1.0.3 unsloth 2025.3.19 unsloth_zoo 2025.3.17 urllib3 2.2.3 uvicorn 0.32.0 virtualenv 20.27.1 wcwidth 0.2.13 websockets 12.0 Werkzeug 3.1.3 wheel 0.44.0 xformers 0.0.29.post3 xxhash 3.5.0 yarl 1.17.1 zipp 3.21.0 zstandard 0.23.0Additional context 使用 unsloth 微调 qwen2.5-vl-7B-instruct 时报错。启动命令：
FPS_MAX_FRAMES=1 FPS=2 CUDA_VISIBLE_DEVICES=0 SIZE_FACTOR=2 MAX_PIXELS=200704 swift sft \ --model_type qwen2_5_vl \ --model Qwen2.5-VL-7B-Instruct \ --tuner_backend unsloth \ --quant_bits 4 \ --num_train_epochs 2 \ --learning_rate 1e-5 \ --lora_rank 128 \ --lora_alpha 64 \ --per_device_train_batch_size 1 \ --per_device_eval_batch_size 1 \ --train_type lora \ --target_modules all-linear \ --max_length 350 \ --gradient_accumulation_steps 16 \ --eval_steps 1000000 \ --save_steps 100 \ --freeze_vit false \ --freeze_llm false \ --split_dataset_ratio 0.001 \ --torch_dtype bfloat16 \ --save_total_limit 2 \ --data_seed 299 \ --dataset clean_jsonl/clean/data.jsonl
参考一下这个issue https://github.com/modelscope/ms-swift/issues/3409
