Mr_Go_is_surfing comments

Repositories
Issues
Comments

Results 11 comments of


                                            Mr_Go_is_surfing

[BUG]: colossal cannot split tensor evenly when using Sequential Parallelism in hybirdplugin

@TongLi3701 Hi! Thanks for your comment. Today I have tried SP mode, and I have tested 4 kinds of sp mode including ring, split_gather, all_to_all，ring_attn. For all_to_all and ring_attn, I...