SwinTransformer and Compact-Transformers Code

[khawar-islam]

Dear @lucidrains

Any hope to implement Swin Transformer and Compact-Transformers code?

Regards, Khawar

[lucidrains]

@khawar512 Hi Khawar, I believe Swin already has a pytorch repository of its own

As for compact transformers, there's really nothing to it - it's just convolutions followed by attention layers

[khawar-islam]

Hi @lucidrains you are right about compact transformer but a compact transformer is very efficient in training. They already sent you a Pull request. If possible for you, please add it into your repo.

[stevenwalton]

Btw, I opened a pull request for compact transformers #128

[khawar-islam]

@stevenwalton that's great thank you