InternEvo icon indicating copy to clipboard operation
InternEvo copied to clipboard

Published 20 hours ago • verified publisher icon InternLM

[Feature] rotary代码规范化

Open sunpengsdu opened this issue 1 year ago • 0 comments
trafficstars

Describe the feature

目前 rotary_embedding类的实现有大量历史遗留的代码，建议和https://github.com/Dao-AILab/flash-attention/blob/v2.2.1/flash_attn/layers/rotary.py 对齐，并且支持triton算子

Will you implement it?

  • [ ] I would like to implement this feature and create a PR!

sunpengsdu avatar Jan 29 '24 12:01 sunpengsdu