Why torch.float32 if args.fp16 ？

Open guihonghao opened this issue 2 years ago • 2 comments

Why torch_dtype=(torch.float32 if args.fp16 else (torch.bfloat16 if args.bf16 else torch.float32)) rather than torch_dtype=(torch.float16 if args.fp16 else (torch.bfloat16 if args.bf16 else torch.float32))？？？

Jul 07 '23 07:07 guihonghao

the same question.

Jul 19 '23 08:07 EachSheep

see #172

Jul 24 '23 14:07 hibagus