Using torch autocast

[AAnoosheh]

Hi,

Can someone confirm if the model is known to work in mixed-precision by simply using torch's autocast function? (or if there's a more proper way intended)

Thanks

[kaihe]

got nan loss trying to use pytorch-lightning mix precision API

trainer_kwargs["precision"] = 16
trainer_kwargs["amp_backend"] = "native"

[joanrod]

@kaihe Did you figure out how to fix the NaN issue? Having the same NaN loss problem.

[kaihe]

@joanrod I give it up mix precision training