pre-training topic

Research code for ECCV 2020 paper "UNITER: UNiversal Image-TExt Representation Learning"

ChenRocks

Conceptual 12M is a dataset containing (image-URL, caption) pairs collected for vision-and-language pre-training.

google-research-datasets

A PyTorch implementation of VIOLET

tsujuifu

OpenAI GPT2 pre-training and sequence prediction implementation in Tensorflow 2.0

akanyaani

💐Kaleido-BERT: Vision-Language Pre-training on Fashion Domain

mczhuge

Code for KDD'20 "Generative Pre-Training of Graph Neural Networks"

acbull

[NeurIPS 2020] "The Lottery Ticket Hypothesis for Pre-trained BERT Networks", Tianlong Chen, Jonathan Frankle, Shiyu Chang, Sijia Liu, Yang Zhang, Zhangyang Wang, Michael Carbin

VITA-Group

Code and Data for EMNLP2020 Paper "KGPT: Knowledge-Grounded Pre-Training for Data-to-Text Generation"

wenhuchen

Probing the representations of Vision Transformers.

sayakpaul

[ICML 2020] DrRepair: Learning to Repair Programs from Error Messages

michiyasunaga