ColossalAI icon indicating copy to clipboard operation
ColossalAI copied to clipboard
verified publisher icon hpcaitech

[FEATURE]: Support SP+PP in Llama etc.

Open GuangyaoZhang opened this issue 1 year ago • 1 comments

Describe the feature

Currently most models like Llama does not support SP together with PP. Please add support for this.

GuangyaoZhang avatar Jun 27 '24 10:06 GuangyaoZhang

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

Title: [FEATURE]: Support SP+PP in Llama etc.

Issues-translate-bot avatar Jun 27 '24 10:06 Issues-translate-bot