Aidyn-A

> I agree on the warning part, additionally we could perhaps format the output as a log and add it to debug logging in apex. I think the log is...

PyTorch stores and saves the weights in default dtype which is FP32 regardless if AMP was used or not. You may try to cast the model to half: ```python import...

Yes, I agree that saving the model in FP16 is risky, so __I do not recommend__ doing it. The accuracy will definitely be worse. I would recommend saving the model...

cc @ngimel

The usage of `aten` in `c10` is not preferred. Would it make more sense to use CUDA Driver API in `c10` like here #94864?

I believe the test failure is unrelated. @lezcano, can you take another look?

@pytorchbot merge