ColossalAI icon indicating copy to clipboard operation
ColossalAI copied to clipboard

Published 20 hours ago • verified publisher icon hpcaitech

[doc] sequence parallel document

Open wangbluo opened this issue 5 months ago • 0 comments

📝 What does this PR do?

Supplementary comparison of the principles of sequence parallel, including ring-attention and Ulysess, and an explanation of their use cases.

wangbluo avatar Sep 27 '24 07:09 wangbluo