NLP
NLP
### 🐛 Describe the bug What is the difference between train.sh and train_sft.sh? use_neft this parameter? the loss of STF should pay attention to the answer(QA) ### Environment _No response_
### 🐛 Describe the bug torchrun --nproc_per_node=8 --nnodes=1 --node_rank=0 --master_addr=localhost --master_port=30013 train.py --pretrained /root/ColossalAI/colossalai/Colossal-LLaMA-2-7b-base --dataset /root/ColossalAI/ColossalAI/applications/Colossal-LLaMA-2/spliced_tokenized_output_arrow/part-00000 /root/ColossalAI/ColossalAI/applications/Colossal-LLaMA-2/spliced_tokenized_output_arrow/part-00001 /root/ColossalAI/ColossalAI/applications/Colossal-LLaMA-2/spliced_tokenized_output_arrow/part-00002 \ /root/ColossalAI/ColossalAI/applications/Colossal-LLaMA-2/spliced_tokenized_output_arrow/part-00003 /root/ColossalAI/ColossalAI/applications/Colossal-LLaMA-2/spliced_tokenized_output_arrow/part-00004 \ /root/ColossalAI/ColossalAI/applications/Colossal-LLaMA-2/spliced_tokenized_output_arrow/part-00005 /root/ColossalAI/ColossalAI/applications/Colossal-LLaMA-2/spliced_tokenized_output_arrow/part-00006 \ /root/ColossalAI/ColossalAI/applications/Colossal-LLaMA-2/spliced_tokenized_output_arrow/part-00007 /root/ColossalAI/ColossalAI/applications/Colossal-LLaMA-2/spliced_tokenized_output_arrow/part-00008 \ /root/ColossalAI/ColossalAI/applications/Colossal-LLaMA-2/spliced_tokenized_output_arrow/part-00009 \...
Collecting package metadata (current_repodata.json): done Solving environment: done # All requested packages already installed. Cloning into 'nccl-fastsocket'... remote: Enumerating objects: 86, done. remote: Counting objects: 100% (86/86), done. remote: Compressing...
I used mlcommons to test the computing power of my machine. Does this open source code support multi-GPU training? I chose the object detection scene，and used the algorithm of Mask...
CUDA_VISIBLE_DEVICES=0 \ python3 llm_sft.py \ --model_type qwen1half-moe-a2_7b-chat \ --model_id_or_path /root/yovole/qwen/Qwen1.5-MoE-A2.7B-Chat \ --sft_type lora \ --tuner_backend swift \ --dtype AUTO \ --output_dir output \ --dataset dureader-robust-zh \ --train_dataset_sample 10000 \ --num_train_epochs...
test -s /root/dlrover/dlrover/go/operator/bin/controller-gen || GOBIN=/root/dlrover/dlrover/go/operator/bin go install sigs.k8s.io/controller-tools/cmd/[email protected] go: sigs.k8s.io/controller-tools/cmd/[email protected]: sigs.k8s.io/controller-tools/cmd/[email protected]: Get "https://proxy.golang.org/sigs.k8s.io/controller-tools/cmd/controller-gen/@v/v0.9.2.info": dial tcp 172.217.163.49:443: i/o timeout make: *** [Makefile:127: /root/dlrover/dlrover/go/operator/bin/controller-gen] Error 1