LLM-Zoo icon indicating copy to clipboard operation
LLM-Zoo copied to clipboard

Published 20 hours ago • verified publisher icon DAMO-NLP-SG
→

Metadata

LLM Zoo collects information of various open- and close-sourced LLMs

🏰 LLM Zoo

As new animal species are being discovered in the world of natural language processing (NLP) 🌍 every day, it becomes necessary to establish a zoo 🦁 to accommodate them.

This project collects below information of various open- and closed-source LLMs (after the release of ChatGPT):

  • Release time
  • Model size
  • Languages supported
  • Domain
  • Training data
  • Links to resources: GitHub, HuggingFace, Demo, Paper, Official blog

📰 News

  • [2023.05.03] First release! We will regularly update 🔄 the repository to keep track of the latest LLMs. We welcome 👐 any contributions to this project. Please feel free to open an issue or submit a pull request to include new LLMs or update the information of existing LLMs 🙏.

📖 Open-Sourced LLMs

Release Time Model Version Size Backbone Langs Domain Training Data GitHub HF Paper Demo Official Blog
2023.02.27 LLaMA llama-7b/13b/33b/65b 7B/13B/33B/65B - en General
detail1T tokens (English CommonCrawl, C4, Github, Wikipedia, Gutenberg and Books3, ArXiv, Stack Exchange)
 [link] [link] [link] - [link]
2023.03.13 Alpaca alpaca-7b/13b 7B/13B LLaMA en General
detail52k instruction-following data generated by InstructGPT [link]
 [link] [link] - [link] [link]
2023.03.13 Vicuna vicuna-7b/13b-delta-v1.1 7B/13B LLaMA en General
detail70K samples from sharedGPT
 [link] [link] - [link] [link]
2023.03.14 ChatGLM chatglm-6b 6B GLM zh, en General
detailsupervised fine-tuning, feedback bootstrap, and reinforcement learning with human feedback
 [link] [link] - - [link]
2023.03.14 ChatGLM chatglm-130b 130B GLM zh, en General
detailsupervised fine-tuning, feedback bootstrap, and reinforcement learning with human feedback
 [link] - [link] [link] [link]
2023.03.16 Guanaco - 7B LLaMA ja, zh, en, de General
detailmultilingual datasets [link]
 [link] [link] - - -
2023.03.24 Dolly dolly-v1-6b 6B GPT-J-6B en General
detail52k stanford alpaca instruction-following data [link]
 - [link] - - [link]
2023.03.24 ChatDoctor - 7B LLaMA en Medicine
detail52K stanford alpaca [link], 100K HealthCareMagic [link], 10K icliniq [link], 5K GenMedGPT-5k [link]
 [link] - [link] [link] -
2023.03.25 LuoTuo Chinese-alpaca-lora 7B LLaMA zh, en General
detailTranslated 52k stanford alpaca instruction-following data [link], guanaco [link]
 [link] [link] - - -
2023.03.26 BELLE BELLE-7B-0.2M/0.6M/1M/2M 7B BLOOMZ-7B1-mt zh, en General
detail0.2M/0.6M/1M/2M Chinese data [link], 52k stanford alpaca instruction-following data [link]
 [link] [link] [link] - -
2023.03.28 Linly (伶荔) Linly-Chinese-LLaMA 7b/13b/33b 7B/13B/33B LLaMA zh General
detailChinese-English parallel corpora [link], Chinese Wikipedia, community interaction, news data [link], scientific literature [link]
 [link] [link] - - -
2023.03.28 Linly (伶荔) Linly-ChatFlow 7b/13b 7B/13B LLaMA zh General
detailBELLE [link], pCLUE [link], CSL [link], GuanacoDataset [link], Chain-of-Thought [link], news_commentary [link], firefly [link]
 [link] [link] - - [link]
2023.04.01 BAIZE baize-7B/13B/30B 7B/13B/30B LLaMA en General
detail52K Stanford Alpaca [link], 54K Quora [link], 57K StackOverFlow [link]
 [link] [link] [link] [link] -
2023.04.03 Koala - 13B LLaMA en General
detailShareGPT, HC3 [link], OIG [link], Stanford alpaca [link], Anthropic HH [link], OpenAI WebGPT [link], OpenAI Summarization [link]
 - [link] - [link] [link]
2023.04.03 BAIZE baize-healthcare-7b 7B LLaMA en Medicine
detail54K Quora [link], 47K medical dialogs [link]
 [link] [link] - - -
2023.04.06 Firefly (流萤) firefly-1b4/2b6 1.4B/2.6B BLOOM-ZH zh General
detailChinese question-answering pairs [link], [link]
 [link] [link] - - -
2023.04.08 Phoenix Phoenix-chat-7b 7B BLOOMZ multi General
detailconversation data [link]
 [link] [link] - - -
2023.04.09 Phoenix Phoenix-inst-chat-7b 7B BLOOMZ multi General
detailconversation data [link], instruction data
 [link] [link] - - -
2023.04.10 Chimera chimera-chat-7b/13b 7B/13B LLaMA latin General
detailconversation data [link]
 [link] [link] - - -
2023.04.11 Chimera chimera-inst-chat-7b/13b 7B/13B LLaMA latin General
detailconversation data [link], instruction data
 [link] [link] - - -
2023.04.12 Dolly dolly-v2-12b 12B pythia-12b en General
detail15k human-generated prompt/response pairs [link]
 [link] [link] - - [link]
2023.04.14 MedAlpaca medalpaca 7b/13b 7B/13B LLaMA en Medicine
detailquestion-answering pairs from flash card, wikidoc, stackexchange and ChatDoctor
 [link] [link] [link] - -
2023.04.19 BELLE BELLE-LLaMA-7B/13B-2M 7B/13B LLaMA zh, en General
detail2M Chinese data [link], 52k stanford alpaca instruction-following data [link]
 [link] [link] [link] - -
2023.04.21 MOSS moss-moon-003-base 16B CodeGen zh, en General
detail100B Chinese tokens and 20B English tokens
 [link] [link] - [link] [link]
2023.04.21 MOSS moss-moon-003-sft 16B moss-moon-003-base zh, en General
detail1.1M multi-turn conversational data (generated from ChatGPT) [link]
 [link] [link] - [link] [link]
2023.04.21 MOSS moss-moon-003-sft-plugin 16B moss-moon-003-base zh, en General
detail1.1M multi-turn conversational data [link], 300K plugin-augmented data (generated by InstructGPT) [link]
 [link] [link] - [link] [link]
2023.04.22 HuggingChat oasst-sft-6-llama-30b 30B LLaMA multi General
detailhuman-generated, human-annotated assistant-style conversation corpus consisting of 161k messages in 35 languages [link]
 [link] [link] - [link] -
2023.06.19 KnowLM zhixi-13b 13B LLaMA zh, en General
detailhuman-generated, machine-generated and Knowledge Graph-generated in Chinese and English [link]
 [link] [link] - - -
2023.06.21 BayLing(百聆) BayLing-7b/13b 7B/13B LLaMA zh, en General
detail160K human-generated, machine-generated multi-turn interactive translation corpus, alpaca instructions and sharegpt conversations [link]
 [link] [link] [link] [link] [link]
2023.07.18 LLaMA 2 llama-2-7b/13b/70b-(chat) 7B/13B/70B - en General
detail2T tokens (Most in English, a new mix of data from publicly available sources)
 [link] [link] [link] - [link]

📕 Closed-Sourced LLMs

Release Time Model Version Size Langs Domain Demo Official Blog Paper
2022.11.30 ChatGPT gpt-3.5-turbo - multi general [link] [link] -
2023.03.14 Claude Claude InstantClaude-v1 - multi general [link] [link] -
2023.03.14 GPT gpt-4 - multi general [link] [link] [link]
2023.03.16 Ernie Bot (文心一言) - - zh, en general [link] [link] -
2023.03.21 Bard - - multi general [link] [link] -
2023.03.30 BloombergGPT - 50B en finance - [link] [link]
2023.04.11 Tongyi Qianwen (通义千问) - - multi general [link] [link] -
2023.07.07 OmModel（欧姆大模型） - - multi general [link] [link] -
2023.07.11 Claude 2 Claude-v2 - multi general - [link] [link]

🏗 TODO List

  • [x] Include open-sourced LLMs
  • [x] Include closed-sourced LLMs
  • [ ] Include a systematic review of common training data
  • [ ] Include interesting use cases of various LLMs
  • [ ] Performance of LLMs on various evaluation tasks

📝 Citation

If you find this repository useful, please consider citing.

@software{li2023llmzoo,
  title = {LLM Zoo}
  author = {Li, Xingxuan and Zhang, Wenxuan and Bing, Lidong},
  url = {https://github.com/DAMO-NLP-SG/LLM-Zoo},
  year = {2023}
}

← Metadata

271
Stars
13
Forks
Watchers

Owner

verified publisher icon DAMO-NLP-SG

Metadata

LLM Zoo collects information of various open- and close-sourced LLMs

Back