ColossalAI icon indicating copy to clipboard operation
ColossalAI copied to clipboard

[FEATURE]: Support qwen2 model

Open wangbluo opened this issue 10 months ago • 0 comments

Describe the feature

We are excited to announce the addition of support for the qwen2 model in the ColossalAI framework. The qwen2 model is compatible with version 4.39.3 of the transformer library, as the transformers of version 4.36 doesn't support qwen2 model.

The qwen2 model inherits most the optimization capabilities of ColossalAI and seamlessly integrates with the transformer library. Users can now utilize the qwen2 model for various tasks such as pretraining and fine-tuning, similar to how they would use other models like llama2.

With the integration of the qwen2 model, ColossalAI continues to empower users with state-of-the-art models and extensive optimization functionalities.

Environment: transformers = 4.39.3

wangbluo avatar Apr 09 '24 09:04 wangbluo