DeepSpeed icon indicating copy to clipboard operation
DeepSpeed copied to clipboard

Recover shared parameters

Open ShijieZZZZ opened this issue 1 year ago • 0 comments

To address this issue:

[BUG] DeepSpeed zero_to_fp32.py script ignores some layers while creating FP32 checkpoints from DS ZeRO checkpoints.

Shared parameters that hold reference are missing when extracts fp32 weights from checkpoint. This PR recovers such shared parameters by linking them with their partners.

ShijieZZZZ avatar Mar 16 '23 00:03 ShijieZZZZ