DeepSpeed icon indicating copy to clipboard operation
DeepSpeed copied to clipboard

Published 20 hours ago • verified publisher icon microsoft

Data efficiency library update

Open conglongli opened this issue 2 years ago • 0 comments

  1. Update blog/tutorial based on new results on BERT pretrain/GPT-2 finetune (corresponding update to examples: https://github.com/microsoft/DeepSpeedExamples/pull/238 and https://github.com/microsoft/Megatron-DeepSpeed/pull/113).
  2. Fix the case when user provides a collate function.
  3. Fix the case when user does not provide a mpu.
  4. Fix checkpoint saving issues when applying the library for MoE model.
  5. Use "/tmp" for unit test.

conglongli avatar Feb 21 '23 09:02 conglongli