[Feature] Torchrun for MPO training

[MathewCrespo]

Motivation

Many of us only have a single node with several GPUs, and it is more common to use torchrun than srun. Hopefully, there will be an official script for MPO training with torchrun.

Related resources

No response

Additional context

No response

[hekaijie123]

@lvhan028 @whai362

[TimeOverflow]

you can just replace srun with torchrun command, just like the fintuning script. it works. see this issue for refrence: https://github.com/OpenGVLab/InternVL/issues/856
No matter torchrun or slurm, just gpu schedule tool, so you can easily move to torchrun.

[Weiyun1025]

You can refer to this script.