ExtremeViscent

[FEATURE]: Customize DDP arguments

1

### Describe the feature In the case when users need to specify DDP launching parameters (e.g.` find_unused_parameters`), keyword arguments should be passed to `torch.nn.parallel.DistributedDataParallel` at initialization.

[FEATURE]: Support for U-Net in pipeline parallel

2

### Describe the feature The pipeline parallel currently only supports sequential models. This restricted models with skip connections like U-Net to perform well. I would like to request an example...

This update allows comm-efficient data parallel based on inherent dimension during classifier-free diffusion (conditional and unconditional). Effects can be tested with scripts/inference_cp.py.