ColossalAI icon indicating copy to clipboard operation
ColossalAI copied to clipboard

Published 20 hours ago • verified publisher icon hpcaitech

[shardformer] Support Qwen2 model

Open wangbluo opened this issue 10 months ago • 0 comments

🚨 Issue number

  • [ ] https://github.com/hpcaitech/ColossalAI/issues/5573

📝 What does this PR do?

[shardformer/modeling/qwen2]: add qwen2.py and qwen2 policy to support qwen2 model, have passed all the tests in test_shard_qwen2.py.

The environment is transformers==4.39.3, as 4.36.0 transformers didn't support qwen2 model.

wangbluo avatar Apr 09 '24 09:04 wangbluo