[FEATURE]: Support for U-Net in pipeline parallel

[ExtremeViscent]

Describe the feature

The pipeline parallel currently only supports sequential models. This restricted models with skip connections like U-Net to perform well. I would like to request an example of pipeline parallel model with non-sequential structure.

[feifeibear]

Thanks for your insights. Can you help with this project? From my point of view, I have not seen real requirements of pipeline parallelism for U-net. So it is not our first priority.

[ChenMinQi]

maybe diffusion model?

[binmakeswell]

We have updated a lot. This issue was closed due to inactivity. Thanks.