modelscope-agent icon indicating copy to clipboard operation
modelscope-agent copied to clipboard

Published 20 hours ago • verified publisher icon modelscope

import flash_attn rotary fail

Open cnahmgx opened this issue 1 year ago • 2 comments

背景：在本地部署ModelScope-Agent-7B，机器为nvidia的A100，速度特别慢，chat一次平均耗时18秒 已经按照[https://modelscope.cn/models/iic/ModelScope-Agent-7B/summary的步骤安装了flash-attention==2.3.5、layer_norm、rotary-embedding-torch==0.5.3] 启动ModelScope-Agent-7B 还是报：Warning: import flash_attn rotary fail, please install FlashAttention rotary to get higher efficiency https://github.com/Dao-AILab/flash-attention/tree/main/csrc/rotary

cnahmgx avatar Jan 30 '24 09:01 cnahmgx

帮忙看看呢

cnahmgx avatar Feb 20 '24 05:02 cnahmgx

您好，最近modelscope-agent更新了版本，建议使用最新版本，关于本地部署的问题，可以参考这个下面这个关于无外网环境部署的方案 https://github.com/modelscope/modelscope-agent/pull/307/files

mushenL avatar Feb 23 '24 03:02 mushenL