PiPPy
PiPPy copied to clipboard
[SPMD][Fusion] - ensure matching ProcessGroups for fused comm calls
Currently we assume all comm calls can be fused with any other comm call (i.e. all use default process group). This is usually correct, but need to implement check of process groups and ensure only same process groups are fused to same process groups.