self-attention topic

list of efficient attention modules

Separius

An ultimately comprehensive paper list of Vision Transformer/Attention, including papers, codes, and related websites

cmhungsteve

Simple Tensorflow Implementation of "A Structured Self-attentive Sentence Embedding" (ICLR 2017)

roomylee

My implementation of the original GAT paper (Veličković et al.). I've additionally included the playground.py file for visualizing the Cora dataset, GAT embeddings, an attention mechanism, and entropy...

gordicaleksa

[TNSRE 2021] "An Attention-based Deep Learning Approach for Sleep Stage Classification with Single-Channel EEG"

emadeldeen24

Datasets, tools, and benchmarks for representation learning of code.

github

Pre-training of Deep Bidirectional Transformers for Language Understanding: pre-train TextCNN

brightmart

Important paper implementations for Question Answering using PyTorch

kushalj001

A PyTorch implementation of Speech Transformer, an End-to-End ASR with Transformer network on Mandarin Chinese.

kaituoxu

The implementation of DeBERTa

microsoft