Open-Sora-Plan icon indicating copy to clipboard operation
Open-Sora-Plan copied to clipboard

Published 20 hours ago • verified publisher icon PKU-YuanGroup

Is DeepSpeed-Ulysses the sequence parallel method used in Open-Sora-Plan v1.2.0?

Open TheTinyTeddy opened this issue 6 months ago • 1 comments

Hi, thank you for the great work!

I was wondering is DeepSpeed-Ulysses the sequence parallel method used in both inference/training of Open-Sora-Plan v1.2.0?

(As a side note, I think you could take a look at the hybrid ring/ulysses method featured in https://github.com/feifeibear/long-context-attention/tree/main )

TheTinyTeddy avatar Aug 01 '24 11:08 TheTinyTeddy