Regarding intuitions behind certain improvements

[tumble-weed]

Hi, thanks for the work! i was interested in certain improvements you had made, and was wondering if you had an insight about why they help. you mention mixed precision training and channels first help. could you help me understand why these have that effect?

thanks

[bberzh]

same confusion,hope your generous answer,thanks!

[dbpprt]

If your hardware has tensor cores then mixed precision and channels last training heavily influences training performance:

• https://pytorch.org/tutorials/intermediate/memory_format_tutorial.html
• https://pytorch.org/blog/accelerating-training-on-nvidia-gpus-with-pytorch-automatic-mixed-precision/

(they do not affect model performance).