NeMo icon indicating copy to clipboard operation
NeMo copied to clipboard
verified publisher icon NVIDIA

FSDP reduce_scatter can not overlap with compute

Open kuangdao opened this issue 1 year ago • 1 comments

I use nemo 24.03 and i find FSDP reduce_scatter can not overlap with compute 截屏2024-06-07 15 24 28

### Tasks

kuangdao avatar Jun 07 '24 07:06 kuangdao

no one concerned about this issue？

kuangdao avatar Jun 18 '24 06:06 kuangdao

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

github-actions[bot] avatar Aug 03 '24 01:08 github-actions[bot]

This issue was closed because it has been inactive for 7 days since being marked as stale.

github-actions[bot] avatar Aug 10 '24 01:08 github-actions[bot]