Shuffle-Transformer

Shuffle-Transformer copied to clipboard

您好，请问在模型实现代码中为什么只需要换一下ws1和hh、ws2和ww的位置就能实现shuffle的操作呢，在括号里相乘(AxB)和(BxA)应该没有什么区别吧，期待您的解答！ 就是下面的代码 q, k, v = rearrange(qkv, 'b (qkv h d) **(ws1 hh) (ws2 ww)** -> qkv (b hh ww) h (ws1 ws2) d', h=self.num_heads, qkv=3, ws1=self.ws, ws2=self.ws) q, k,...

endaoguansanlu

这个有C++部署代码吗？

1

非常感谢您的开源，请问有C++部署代码？

longzeyilang

get_sinusoid_encoding()

1

if self.has_pos_embed: self.pos_embed = nn.Parameter(data=get_sinusoid_encoding(n_position=num_patches, d_hid=embed_dim), requires_grad=False) self.pos_drop = nn.Dropout(p=drop_rate) Hello, is get_sinusoid_encoding used? Can I remove it if I don't use it? This function doesn't seem to be defined.

Dpw506

I write a shuffle-large config following swin-large and training on ImageNet22K dataset using apex O1. But the training process is unstable and the loss quickly become NAN. Is there any...

jiandan42