Phillip Lippe

Hi Miltos, can you try to first merge the current master into your branch? Currently, there are merge conflicts in your notebooks.

@jerronl That's a very good point, I didn't think of that before. Let me check if I can add a default background color to all SVG images.

Thanks again for pointing that out. Added background color to all images in a2fe2d3cad57fc56e6898d30c8bbd78b3e0a1c0e. Let me know in case you spot any that images that are still hard to read...

Hi, thanks for opening the pull request. I am aware of the deprecation warning, but actively decided not to change it. The reason is that the solution you also have...

We mostly kept it in because in the lectures of our Deep Learning course, we give a bit of an overview of the different CNNs that were proposed over time,...

In first checks, PyTorch 2.0 is a bit faster than PyTorch 1, but doesn't reach the performance of JAX, e.g., on the Transformer models. I noticed that it can give...

Hi @pi-tau, in general, the models presented here are based on the common 'modern' implementations of them and not necessarily 1-to-1 as in the original papers. Regarding your specific questions:...

Thanks!

Hi @pi-tau, when you use Adam, the gradients are scaled back to a unit norm, such that multiplying the loss by a constant does not make a difference. This is...

If you train with SGD, then you either have to adjust your learning rate and grad norm, or scale the loss back to typical values, e.g. by taking the mean...