Chien-Chin Huang comments

Results 119 comments of


                                            Chien-Chin Huang

Checkpoint saves failing for eager mode training

During this step, only rank0 is doing some reduction work of the plans. But I'm surprised it will be slow enough to cause the NCCL timeout. Verifying with a large...

[PT2D] Ensure the trace rules are correct with distributed

@pytorchbot merge

[DCP] Always unflatten containers even if no tensors present

@pytorchbot merge -f "The failing tests are not related"

[DCP] Always create requests for non-tensor objects

@pytorchbot merge -f "The failing tests are not related."

[DSD] Correctly handle _extra_state

@pytorchbot merge -f "The failing tests are not related."

[FSDP][optim_state_dict] Let FSDP support NamedOptimizer/KeyedOptimizer when use_orig_params is False

Will land this PR to unblock the use case of NamedOptimizer. This will also include the support of `NamedOptimizer` + `use_orig_params=False`. We can remove the part to support `use_orig_param=False` later...

[FSDP][optim_state_dict] Let FSDP support NamedOptimizer/KeyedOptimizer when use_orig_params is False

The failing tests are not related.

[FSDP][optim_state_dict] Let FSDP support NamedOptimizer/KeyedOptimizer when use_orig_params is False

@pytorchbot merge -f "The failing tests are not related"

[FSDP][optim_state_dict][9/N] Rewrite the all-gather flow of optimizer state to support older GPUs

@pytorchbot merge

[FSDP][optim_state_dict][9/N] Rewrite the all-gather flow of optimizer state to support older GPUs

@pytorchbot merge