llm-papers

List of Large Lanugage Model Papers

GPTs by OpenAI

• GPT-1: Improving Language Understanding by Generative Pre-Training (2018)
• GPT-2: Language Models are Unsupervised Multitask Learners (2019)
• GPT-3: Language Models are Few-Shot Learners (2020)
• InstructGPT: Training language models to follow instructions with human feedback (2022)
• ChatGPT: Introducing ChatGPT, blog (2022)
• GPT-4: GPT-4 Technical Report (2023)

Prompt

• Chain-of-Thought: Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (Google, NeurIPS, 2022)
• ReAct: REACT: SYNERGIZING REASONING AND ACTING IN LANGUAGE MODELS (Google, ICLR, 2023)
• Self-Ask: MEASURING AND NARROWING THE COMPOSITIONALITY GAP IN LANGUAGE MODELS (UW, 2023)

Finetune

• Prompt Tuning: The Power of Scale for Parameter-Efficient Prompt Tuning (Google, EMNLP, 2021)
• Prefix Tuning: Prefix-Tuning: Optimizing Continuous Prompts for Generation (Stanford, IJCNLP, 2021)
• LoRA: LoRA: Low-Rank Adaptation of Large Language Models (Microsoft, ICLR, 2022)
• P-Tuning: P-Tuning: Prompt Tuning Can Be Comparable to Fine-tuning Across Scales and Tasks (Tsinghua, ACL, 2022)
• P-Tuning v2: P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks (Tsinghua, ACL, 2022)
• AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning (Georgia Tech, ICLR, 2023)
• QLoRA: QLoRA: Efficient Finetuning of Quantized LLMs (UW, Submitted to NeurIPS, 2023)

Multi Modality

Image

• BLIP-2: BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models (Salesforce, 2023.01)
• PaLM-E: PaLM-E: An Embodied Multimodal Language Model (Google, 2023.03)
• LLaVA: Visual Instruction Tuning (Microsoft, 2023.04), github
• MiniGPT-4: MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models (KAUST, 2023.04)
• mPLUG-Owl: mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality (Alibaba, 2023.04)
• InstructBLIP: InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning (Salesforce, 2023.05)

Speech

• AudioGPT: AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head (ZJU, 2023.04, github)
• SpeechGPT: SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilities (FUDAN, 2023.05, github)