transformers_tasks icon indicating copy to clipboard operation
transformers_tasks copied to clipboard
Published 2 years ago • verified publisher icon HarderThenHarder

--mixed_precision=fp16

Open WGS-note opened this issue 2 years ago • 2 comments

多卡训练需要去掉混合精度，否则会：RuntimeError: expected scalar type Half but found Float，因为模型是Half加载的

WGS-note avatar May 11 '23 02:05 WGS-note

用p-tuning还是lora做的，lora多卡我都跑不起来，量化直接报错

starphantom666 avatar May 11 '23 03:05 starphantom666

我没有用--mixed_precision=fp16没有报错

pengcao avatar Sep 02 '23 07:09 pengcao