DeepSpeedExamples icon indicating copy to clipboard operation
DeepSpeedExamples copied to clipboard
verified publisher icon microsoft

No IGNORE_INDEX for sft

Open payne53 opened this issue 2 years ago • 0 comments

https://github.com/microsoft/DeepSpeedExamples/blob/7eac9f699442fbc3f96b2dbdb2432d3847406a47/applications/DeepSpeed-Chat/training/utils/data/data_utils.py#L126 for stage 1 sft, the labels do not add IGNORE_INDEX for the prompt, should this be right?

payne53 avatar Apr 19 '23 11:04 payne53