Dong-Hyun Lee
Dong-Hyun Lee
### Non-arxiv - **How should AI systems behave, and who should decide?** by OpenAI (16 Feb 2023) - https://openai.com/blog/how-should-ai-systems-behave/ - ChatGPT는 어떻게 운영되는가? - A two step process : pre-training,...
### ChatGPT Is a Blurry JPEG of the Web (9 Feb, 2023) https://www.newyorker.com/tech/annals-of-technology/chatgpt-is-a-blurry-jpeg-of-the-web - Kyunghyun Cho Twit 에서 추천 (https://twitter.com/kchonyc/status/1624633568734355456) 예전 Cho 발표 내용과 비슷한 관점 (https://drive.google.com/file/d/1dk3o-fcdH1Y7-rGGqlVR35AZ1CVwz0qi/view) - LLM은 Web에...
### Scaling Instruction-Finetuned Language Models (Arxiv 2022 Dec 6, Google) - Instruction (Multi-task) Fine-Tuning on PaLM (8B, 62B, 540B) and T5 (80M~11B) : **Flan-PaLM**, **Flan-T5** - Task Scaling up to...
### Accelerating Large Language Model Decoding with Speculative Sampling - LLM의 Sampling latency를 줄이려는 노력 (2~2.5x decoding speedup) - 훨씬 가벼운 draft model이 길이 K의 짧은 draft 를 생성하게 한...
- [Constitutional AI: Harmlessness from AI Feedback](https://arxiv.org/abs/2212.08073) - 최근에 ChatGPT에 썼다고 해서 핫한 RLHF (LM의 응답들에 대한 사람의 선호도를 기반으로 언어모델 강화학습) - Human Feedback으로는 Human Performance를 넘을 수 없다?!...
## Non-Arxiv #### Stanford Alpaca: An Instruction-following LLaMA Model * https://github.com/tatsu-lab/stanford_alpaca * fine-tuned from a 7B LLaMA model on 52K instruction-following data generated by the techniques in the Self-Instruct paper...