Szymon
Szymon
## Evaluation Dilemma - Benchmark dataset may have been collected in pre-training/post-training, which make it can't evaluate LLM properly - LLM could involve with specific data, evaluation only take place...
```html body { background-image: url('https://boson.ai/bg.jpg'); background-attachment: fixed; background-size: cover; } .header { padding-top: 50px; margin: auto; width: 60%; font-family: Arial,Helvetica,sans-serif; } h1 { font-size: 110px; color: #fff; } h2 {...
## Open Multi-Modal LLMs - https://github.com/THUDM/VisualGLM-6B - https://github.com/DAMO-NLP-SG/Video-LLaMA - https://github.com/melodysdreamj/WizardVicunaLM
https://lilianweng.github.io/posts/2023-06-23-agent/ ## Tool Learning ### Plugins - Toolformer: [Toolformer: Language Models Can Teach Themselves to Use Tools, 2023/02, Meta AI] - https://github.com/conceptofmind/toolformer - https://github.com/TeamDman/toolformer - GPT4 3rd Party Plugins: https://openai.com/blog/chatgpt-plugins...
## Synthetic Data Generation - LLM ICL Distillation: https://www.amazon.science/blog/using-large-language-models-llms-to-synthesize-training-data | [Does Synthetic Data Generation of LLMs Help Clinical Text Mining?] https://arxiv.org/abs/2303.04360 - [SELF-INSTRUCT: Aligning Language Models with Self-Generated Instructions ,...
| 序号 | 模型名 | |-----|--------| | 1 | 科大讯飞 星火 | | 2 | 印象笔记 大象GPT | | 3 | 第四范式 式说 | | 4 | 数慧时空 长城 |...
### [Instruction tuning with GPT-4, Microsoft, 2023.04] - This paper intent to build the first Self-Instruct LLM using GPT-4 response based on LLaMA-7B. [blog](https://instruction-tuning-with-gpt-4.github.io/). - 1st, it collect 52K English...
### [PaLM 2 Technical Report, Google, 2023.05] - scaling law: power law to equal proportion(1:1), find out data size is at least as important as model size; data selection and...
### [GPT-4 Technical Report, OpenAI, 2023.03] - **no further details** about architecture (including model size), hardware, training compute, dataset construction, traning method, or similar. - Multi-modal: accept image and text...
### [Sparks of Artificial General Intelligence: Early experiments with GPT-4, MSFT, 2023.04] - (1) refine: refined over span of a month - (2) Multimodal and interdisciplinary composition: not only does...