davidie

lora权重文件有14G

2

基座模型选的BELLE-7B-2M，进行LoRA微调，使用src/train.py torchrun --nproc_per_node 8 train.py \ --model_name_or_path ${model_name_or_path} \ --deepspeed configs/deepspeed_config_stage3.json \ --use_lora True \ --lora_config configs/lora_config_bloom.json \ --train_file ${train_file} \ --validation_file ${validation_file} \ --per_device_train_batch_size 2 \ --per_device_eval_batch_size 2 \...