InternEvo icon indicating copy to clipboard operation
InternEvo copied to clipboard

Published 20 hours ago • verified publisher icon InternLM

[Bug] AssertionError: Only flash cross entropy support parallel_output

Open wen020 opened this issue 1 year ago • 1 comments
trafficstars

Describe the bug

image 已经按照安装指引安装好了环境，在pretrain的时候报这个错误

Environment

环境完全按照 https://github.com/InternLM/InternEvo/blob/develop/doc/install.md 安装

Other information

No response

wen020 avatar Jun 06 '24 00:06 wen020

你这个是用了flash attention还是没用呢，如果不用fa的话就把config里parallel_output设置为False。如果是要用的话看起来是FA没装上，import flash_attn看看什么结果

li126com avatar Jun 14 '24 03:06 li126com