Phil Wang

https://github.com/lucidrains/byol-pytorch/tree/0.5.1#advanced I've added the ability to do so in 0.5.1!

@shivam-aggarwal can you give DDP a try? I think DDP is the preferred way to go distributed these days anyhow

@sdtblck if this still doesn't work, we'll give the non-cached version on the same branch a try

@sdtblck hmm, i don't see the logits mask being used at all, so it's likely going to be still broken with this old way of doing things

@arogozhnikov what would it take for you to build this out to your heart's content? you are the only one in the world who can do this justice, imo

i'm convinced that works of art like einops can't be extrinsically motivated into existence, but if Alex wants to put up a Patreon, would be glad to become a patron...

oh. my. god. https://github.com/arogozhnikov/eindex it's happening

@Mut1nyJD thanks! the more i think about this paper, the more excited i am thank you for bringing this to my attention :smile:

@Mut1nyJD ohh yes, i remember this paper, though not as excited about that than the guidance on self-attention map

@Mut1nyJD from that group, i am most excited about https://arxiv.org/abs/2209.15171, will be circling around back to bio soon