weikeltf

Issue:"Your setup doesn't support bf16/gpu. You need torch>=1.10, using Ampere GPU with cuda>=11.0"

1

My GPU is 8*TeslsV100 32G, the software environment is python3.10+cuda11.6+torch2.0.0+ transformers4.28.0.dev0, I run the fine tuning code: torchrun --nproc_per_node=8 --master_port=20001 fastchat/train/train_mem.py \ --model_name_or_path /llama-13b \ --data_path alpaca-data-conversation.json \ --bf16 True...

RuntimeError: Error(s) in loading state_dict for RWKV: Unexpected key(s) in state_dict: "blocks.24.ln1.weight",.....................