Phil Wang

Implementation of E(n)-Equivariant Graph Neural Networks, in Pytorch

lucidrains

A simple and working implementation of Electra, the fastest way to pretrain language models from scratch, in Pytorch

lucidrains

Pytorch implementation of SIREN - Implicit Neural Representations with Periodic Activation Function

lucidrains

Implementation of SE3-Transformers for Equivariant Self-Attention, in Pytorch. This specific repository is geared towards integration with eventual Alphafold2 replication.

lucidrains

Implementation of E(n)-Transformer, which incorporates attention mechanisms into Welling's E(n)-Equivariant Graph Neural Network

lucidrains

Implementation of the Transformer variant proposed in "Transformer Quality in Linear Time"

lucidrains

Implementation of a memory efficient multi-head attention as proposed in the paper, "Self-attention Does Not Need O(n²) Memory"

lucidrains

Implementation of Enformer, Deepmind's attention network for predicting gene expression, in Pytorch

lucidrains

Implementation of 'lightweight' GAN, proposed in ICLR 2021, in Pytorch. High resolution image generations that can be trained within a day or two

lucidrains

Implementation of the 😇 Attention layer from the paper, Scaling Local Self-Attention For Parameter Efficient Visual Backbones

lucidrains