pretrained-language-model topic

A curated list of pretrained sentence and word embedding models

Separius

word2vec, sentence2vec, machine reading comprehension, dialog system, text classification, pretrained language model (i.e., XLNet, BERT, ELMo, GPT), sequence labeling, information retrieval, informati...

gaoisbest

Code associated with the Don't Stop Pretraining ACL 2020 paper

allenai

An optimized deep prompt tuning strategy comparable to fine-tuning across scales and tasks

THUDM

A plug-and-play library for parameter-efficient-tuning (Delta Tuning)

thunlp

Worth-reading papers and related resources on attention mechanism, Transformer and pretrained language model (PLM) such as BERT. 值得一读的注意力机制、Transformer和预训练语言模型论文与相关资源集合

ZhengZixiang

[NeurIPS 2021] COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining

microsoft

MWPToolkit is an open-source framework for math word problem(MWP) solvers.

LYH-YF

This repository contains the code for the paper in Findings of EMNLP 2021: "EfficientBERT: Progressively Searching Multilayer Perceptron via Warm-up Knowledge Distillation".

cheneydon

Foundation Architecture for (M)LLMs

microsoft