Chien-Chin Huang comments

Results 119 comments of


                                            Chien-Chin Huang

[DSD] Implement broadcast_from_rank0 option for optim state_dict

@pytorchbot merge

[DSD] Implement broadcast_from_rank0 option for optim state_dict

@pytorchbot merge -f "The failing tests are not related."

[DSD] Implement broadcast_from_rank0 option for model state_dict

@pytorchbot merge

[DSD] Implement broadcast_from_rank0 option for model state_dict

@pytorchbot merge

[DCP] Introduce async staging extension points

The only concern of merging async_save with save is the growing arguments. Given that we are adding extra arguments (e.g., `executor`) to `async_save`, that there will be some arguments for...

[DCP] DCP load for non-tensor values

cc., @LucasLLC

[DCP] DCP load for non-tensor values

@LucasLLC Did we get the diff in 2.4.1, if not should we aim for 2.4.2?

[DCP] DCP load for non-tensor values

cc., @pradeepfn

reload existing llama checkpoints

DCP has the format util to help the conversion. However, HF conversion should not live in PyTorch code base.

[v.2.4.0] Release Tracker

Link to landed trunk PR (if applicable): * https://github.com/pytorch/pytorch/pull/128685 Link to release branch PR: * https://github.com/pytorch/pytorch/pull/129252 Criteria Category: * Critical fixes to crashes for loading distributed optimizer state_dict with shared...