BELLE icon indicating copy to clipboard operation
BELLE copied to clipboard

Published 20 hours ago • verified publisher icon LianjiaTech

谁能帮忙解决呀，都是用的官方的docker;File "/opt/conda/lib/python3.8/site-packages/deepspeed/runtime/zero/stage3.py", line 133, in __init__ self.dtype = self.optimizer.param_groups[0]['params'][0].dtype

Open listwebit opened this issue 2 years ago • 3 comments
trafficstars

35

listwebit avatar Apr 23 '23 12:04 listwebit

35

您运行的是哪个脚本

xianghuisun avatar Apr 23 '23 13:04 xianghuisun

35

如果把transformers版本降低为4.28.1试下呢

xianghuisun avatar Apr 23 '23 13:04 xianghuisun

确认了一下，可docker镜像没有关系。

bestpredicts avatar Apr 23 '23 14:04 bestpredicts

运行的是bash training_scripts/single_node/run_Loro.sh

listwebit avatar Apr 24 '23 00:04 listwebit

同问

derrickcyt avatar Apr 24 '23 04:04 derrickcyt

transformers==4.28.1 也遇到了相同问题

AlexXx-Wu avatar Apr 24 '23 08:04 AlexXx-Wu

35

我们已经更新了LoRA训练的代码。

xianghuisun avatar Apr 24 '23 12:04 xianghuisun