LambdaNetworks

LambdaNetworks copied to clipboard

in line 44 `lambda_p = F.conv3d(values, self.embedding, padding=(0, self.padding, self.padding))` in `model.py` Is there any reason using conv3d? Can it be implemented using conv2d? Thank you.

cmhungsteve

Hi, @leaderj1001 Thanks for your code. However, I am a little bit confused about this line https://github.com/leaderj1001/LambdaNetworks/blob/3a51b6a7dbac9490dffed6a1bf68157e93135da6/model.py#L48 It seems like did not use relative position embedding when strategy is not...

BrandonHanx

Using BarchNorm2 Instead of softmax operation

1

ssram50

Wall-clock performance issues

2

LambdaResNet50 in comparison to a normal ResNet50 from my testing is 7x slower. I am training on ImageNet, and I am comparing your LambdaResNet50 to a torch.hub ResNet50. Is this...

verbiiyo