Shahar Papini

Move transformer prenorm on the residual path

6

## Pull Request Template ### Checklist - [X] Confirmed that `run-checks all` script has been executed. - [X] Made sure the book is up to date with changes in this...

Transformer prenorm location

1

### Feature description Currentlt, the transformer module has a `norm_first` flag, but I think it is not used as intended. Currently, is does this: ```x=norm(x); x += FF(x)``` According to...

https://github.com/preritj/progressive_growing_of_GANs/blob/04351d69b2e8e9616715f3ac8cb155488953777d/model.py#L85 I think there is a bug here, or at least an implementation of gradient penalty not consistent with the original paper. the alpha coefficient should be randomized per image,...