LLaMA-Factory icon indicating copy to clipboard operation
LLaMA-Factory copied to clipboard

尊敬的作者您好,我在对baichuan-7b进行sft训练时 使用chatglm-Efficient-Tuning项目下的covid\train.json文件进行训练,出现如下提示是否正常?

Open BoFan-tunning opened this issue 2 years ago • 0 comments

1、dataset_info.json文件核对后发现现在的和原项目的一致,但是我把因为将原文件内容只保留5分之1因此,将 "file_sha1": "647f4ad447bd993e4b6b6223d1be15208bab694a"删了内容如下,应该没有问题吧?

"covid_train": { "file_name": "train.json",
"columns": { "prompt": "instruction", "query": "input", "response": "output", "history": "history" } }, 2、训练开始前的部分提示,是否正常? 06/26/2023 18:29:48 - INFO - utils.common - Loading dataset train.json... 06/26/2023 18:29:48 - WARNING - utils.common - Checksum failed: missing SHA-1 hash value in dataset_info.json or too many files. 06/26/2023 18:29:49 - INFO - datasets.builder - Using custom data configuration default-66cdf391fa29513b

Running tokenizer on dataset: 0%| | 0/11466 [00:00<?, ? examples/s]06/26/2023 18:31:57 - INFO - datasets.arrow_dataset - Caching processed dataset at C:\Users\fbai7.cache\huggingface\datasets\json\default-66cdf391fa29513b\0.0.0\e347ab1c932092252e717ff3f949105a4dd28b27e842dd53157d2f72e276c2e4\cache-8cb655356bcb99fc.arrow input_ids: [703, 9805, 1493, 650, 15695, 3475, 680, 755, 1313, 9715, 945, 5840, 9091, 79, 776, 9091, 4278, 8922, 31125, 7320, 31125, 680, 1265, 952, 8943, 678, 656, 3475, 31155, 31114, 3155, 79, 5, 20250, 31143, 31106, 4384, 53, 19626, 6347, 72, 2885, 32027, 32359, 31407, 31920, 31498, 31407, 31323, 72, 32027, 32359, 31350, 32220, 30555, 72, 33712, 33388, 4071, 31892, 31991, 72, 8885, 26249, 31604, 31737, 31183, 31642, 2885, 35310, 31323, 8568, 31146, 31883, 13595, 31183, 5305, 31347, 14780, 31298, 31939, 31407, 31323, 31717, 35310, 33022, 31145, 72, 31717, 32046, 31265, 12862, 4538, 73, 31488, 31308, 77, 53, 76, 2964, 21901, 4531, 7569, 81, 6292, 8831, 17492, 72, 4531, 31433, 4465, 21901, 31192, 32186, 32394, 31399, 28357, 33712, 33388, 4071, 31991, 31305, 4368, 31989, 31306, 31309, 32046, 31265, 4538, 81, 54, 76, 2838, 6076, 31343, 5972, 31989, 31306, 31309, 4640, 3238, 81, 5, 7905, 18056, 31143, 31106, 1, 31106, 35310, 33022, 31405, 31372, 19085, 31172, 75, 31601, 31825, 26796, 75, 2] inputs: A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions. Human: 今年1月初以来,出现左侧口角流口水,左侧手脚乏力,吞咽功能减退,尤其是喝水吃药时易出现呛水现象(晚睡觉时也会因口腔内含口水致呛咳),致肺部反复感染。请教:1、目前用药是否合理?针对当前病情,是否需增加用药来 延缓或阻止吞咽功能退化导致吸入性肺部感染?2、如何避免及减少吸入性肺炎发生? Assistant: 呛咳病程多久了?神志清醒? label_ids: [-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 31106, 35310, 33022, 31405, 31372, 19085, 31172, 75, 31601, 31825, 26796, 75, 2] labels: 呛咳病程多久了?神志清醒? 3、如下警告如何处置,还是可以不用管? 06/26/2023 18:29:48 - WARNING - utils.common - Please specify prompt_template if you are using other pre-trained models. 我理解我的pre-trained models就是百川,prompt_template是不是就是如下参数?

CUDA_VISIBLE_DEVICES=0 python src/train_sft.py
--model_name_or_path path_to_your_model
--do_train
--dataset alpaca_gpt4_en
--finetuning_type lora
--output_dir path_to_sft_checkpoint
--overwrite_cache
--per_device_train_batch_size 4
--gradient_accumulation_steps 4
--lr_scheduler_type cosine
--logging_steps 10
--save_steps 1000
--learning_rate 5e-5
--num_train_epochs 3.0
--plot_loss
--fp16

BoFan-tunning avatar Jun 26 '23 11:06 BoFan-tunning