Megatron-LM
Megatron-LM copied to clipboard
[QUESTION]Is there any plan to make custom_fsdp compatible with PP?
- Tensor Parallelism (TP), Expert Parallelism (EP) and Context Parallelism (CP): Compatible with TP, EP and CP configurations, enabling efficient scaling of large language models.
only shows support TP,EP, and CP in the custom_fsdp.md doc Is there a plan to make custom_fsdp compatible with PP?
Marking as stale. No activity in 60 days.