[feature request] OneStepOff Trainer

[syeehyn]

Creating this Issue and I may claim it once I do have time.

context: our training environments suffered from long tail rollout time, OneStepOff may be a good solution here https://verl.readthedocs.io/en/latest/advance/one_step_off.html

[ultmaster]

I heard that the people from Tencent are implementing ROLL on top of Agent-lightning. I'm sharing the information here and maybe you guys can collaborate.

[yanfei-zhang-95]

I am also interested in this issue, make a note.