long-context-attention icon indicating copy to clipboard operation
long-context-attention copied to clipboard
Published 2 months ago • verified publisher icon feifeibear

Is there example of how to use the hybrid-sp in Megatron-LM?

Open xs1997zju opened this issue 1 year ago • 1 comments

Is there example of how to use the hybrid-sp in Megatron-LM?

xs1997zju avatar Jul 22 '24 08:07 xs1997zju

Could you please refer to this PR in FlagScale, which is a framework built based on Megatron-LM.

https://github.com/FlagOpen/FlagScale/pull/156

feifeibear avatar Jul 23 '24 01:07 feifeibear