Chinese-Transformer-XL icon indicating copy to clipboard operation
Chinese-Transformer-XL copied to clipboard
verified publisher icon THUDM

微调报错 AttributeError: 'DeepSpeedZeroOptimizer' object has no attribute '_restore_from_fp16_weights'

Open binggoml opened this issue 4 years ago • 1 comments

AttributeError: 'DeepSpeedZeroOptimizer' object has no attribute '_restore_from_fp16_weights'

这是怎么回事呀？？

binggoml avatar Feb 17 '22 04:02 binggoml

这个问题应该是新版本的DeepSpeed将_restore_from_fp16_weights重命名为了_restore_from_bit16_weights，但同时在DeepSpeedEngine中没有修改。一种解决方法是使用旧版本的DeepSpeed（我们使用的是0.3.16），另一种是修改deepspeed/runtime/engine.py，将self.optimizer._restore_from_fp16_weights()这行修改为self.optimizer._restore_from_bit16_weights()

duzx16 avatar Feb 22 '22 09:02 duzx16