Megatron-LM icon indicating copy to clipboard operation
Megatron-LM copied to clipboard
Published 1 month ago • verified publisher icon NVIDIA

[QUESTION]Is there any plan to make custom_fsdp compatible with PP?

Open XCD4P opened this issue 10 months ago • 1 comments

  • Tensor Parallelism (TP), Expert Parallelism (EP) and Context Parallelism (CP): Compatible with TP, EP and CP configurations, enabling efficient scaling of large language models.

only shows support TP,EP, and CP in the custom_fsdp.md doc Is there a plan to make custom_fsdp compatible with PP?

XCD4P avatar Mar 06 '25 15:03 XCD4P

Marking as stale. No activity in 60 days.

github-actions[bot] avatar May 05 '25 18:05 github-actions[bot]