pretrain topic

Paper Lists, Notes and Slides, Focus on NLP. For summarization, please refer to https://github.com/xcfcode/Summarization-Papers

xcfcode

大规模中文自然语言处理语料 Large Scale Chinese Corpus for NLP

brightmart

An official implementation for " UniVL: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generation"

microsoft

BERT-CCPoem is an BERT-based pre-trained model particularly for Chinese classical poetry

THUNLP-AIPoet

Large-scale Pre-training Corpus for Chinese 100G 中文预训练语料

CLUEbenchmark

MatDGL is a neural network package that allows researchers to train custom models for crystal modeling tasks. It aims to accelerate the research and application of material science.

huzongxiang

Bert-based models(BERT, MTB, CP) for relation extraction.

thunlp

ALBERT trained on Mongolian text corpus

bayartsogt-ya

[ICLR'23 Spotlight🔥] The first successful BERT/MAE-style pretraining on any convolutional network; Pytorch impl. of "Designing BERT for Convolutional Networks: Sparse and Hierarchical Masked Modeling...

keyu-tian

code for paper "Masked Frequency Modeling for Self-Supervised Visual Pre-Training" (https://arxiv.org/pdf/2206.07706.pdf)

CoinCheung