Phil Wang

Implementation of the Belief State Encoder / Decoder in the new breakthrough robotics paper from ETH Zürich

lucidrains

A Pytorch implementation of Attention on Attention module (both self and guided variants), for Visual Question Answering

lucidrains

Implementation of AudioLM, a SOTA Language Modeling Approach to Audio Generation out of Google Research, in Pytorch

lucidrains

Axial Positional Embedding for Pytorch

lucidrains

A simple cross attention that updates both the source and target in one step

lucidrains

Implementation of COCO-LM, Correcting and Contrasting Text Sequences for Language Model Pretraining, in Pytorch

lucidrains

Implementation of "compositional attention" from MILA, a multi-head attention variant that is reframed as a two-step attention process with disentangled search and retrieval head aggregation, in Pytor...

lucidrains

Implementation of Cross Transformer for spatially-aware few-shot transfer, in Pytorch

lucidrains

Implementation of the DDPM + IPA (invariant point attention) for protein generation, as outlined in the paper "Protein Structure and Sequence Generation with Equivariant Denoising Diffusion Probabilis...

lucidrains

Implementation of the retriever distillation procedure as outlined in the paper "Distilling Knowledge from Reader to Retriever"

lucidrains