pre-training topic

[ICCV 2021 Oral + TPAMI] Just Ask: Learning to Answer Questions from Millions of Narrated Videos

antoyang

Pre-training of Deep Bidirectional Transformers for Language Understanding: pre-train TextCNN

brightmart

Open Source Pre-training Model Framework in PyTorch & Pre-trained Model Zoo

dbiir

Oscar and VinVL

microsoft

Awesome list for research on CLIP (Contrastive Language-Image Pre-Training).

yzhuoning

[NeurIPS 2020] "Graph Contrastive Learning with Augmentations" by Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, Yang Shen

Shen-Lab

GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training @ KDD 2020

THUDM

Papers about pretraining and self-supervised learning on Graph Neural Networks (GNN).

ChandlerBang

Code for TKDE paper "Self-supervised learning on graphs: Contrastive, generative, or predictive"

LirongWu

Code for ICLR 2020 paper "VL-BERT: Pre-training of Generic Visual-Linguistic Representations".

jackroos