DeepSpeedExamples icon indicating copy to clipboard operation
DeepSpeedExamples copied to clipboard

Published 20 hours ago • verified publisher icon microsoft

no "--deepspeed" in step2 shell

Open ucas010 opened this issue 2 years ago • 1 comments

can't find the "--deepspeed" in main.py but find in shell(training/step2_reward_model_finetuning/training_scripts/single_gpu/run_350m.sh)

deepspeed --num_gpus 1 main.py --model_name_or_path facebook/opt-350m \
   --num_padding_at_beginning 1 --gradient_accumulation_steps 2 --zero_stage $ZERO_STAGE \
   --only_optimize_lora \ 
   --deepspeed --output_dir $OUTPUT &> $OUTPUT/training.log \

ucas010 avatar Apr 20 '23 06:04 ucas010

@ucas010 --deepspeed is consumed by DeepSpeed via https://github.com/microsoft/DeepSpeedExamples/blob/3a77097a7b6ce67b78ea7c05129b90915cfebbb0/applications/DeepSpeed-Chat/training/step2_reward_model_finetuning/main.py#L165

Read more here: https://deepspeed.readthedocs.io/en/latest/initialize.html#deepspeed.add_config_arguments

mrwyattii avatar Apr 21 '23 23:04 mrwyattii