Junfeng Tian comments

Repositories
Issues
Comments

Results 2 comments of


                                            Junfeng Tian

[REQUEST] universal checkpoint for ZeRO - 1,2,3

Instead of the universal checkpoint, I use the code from @tjruwase to convert a DeepSpeed checkpoint without TP and PP (128 ranks) to another DeepSpeed checkpoint (32 ranks). https://gist.github.com/rgtjf/aa90fc37efe38ad773046623780a1026 Discussions...

Junfeng Tian

[REQUEST] universal checkpoint for ZeRO - 1,2,3

Pre-exacted image features