Question about FFN

[WandernForte]

trafficstars

do the following two version'result the same? if not, is it because of different sequences of norm2 and mlp? 这两种版本的结果是否相同？如果不同的话，是因为norm2和mlp的调用顺序不同吗？

# FFN-swin-V2
        x = x + self.drop_path(self.norm2(self.mlp(x)))
# FFN-swin
        x = x + self.drop_path(self.mlp(self.norm2(x)))

I'm trying to implements swinv2 used for semantic segmentation in mmlab(not officially), their imple of swin was different from your original though. 我正在尝试在mmlab框架上实现用于语义分割的swinv2, 他们实现的swin和源实现是不同的。

[ancientmooner]

Swin V1 uses pre-norm layers. Swin V2 uses a new normalization configuration named res-post-norm. Please look into https://arxiv.org/pdf/2111.09883.pdf for details.