NeMo
NeMo copied to clipboard
FSDP reduce_scatter can not overlap with compute
I use nemo 24.03 and i find FSDP reduce_scatter can not overlap with compute
### Tasks
no one concerned about this issue?
This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.
This issue was closed because it has been inactive for 7 days since being marked as stale.