Chinese-Vicuna icon indicating copy to clipboard operation
Chinese-Vicuna copied to clipboard

Published 20 hours ago • verified publisher icon Facico

多轮对话导致OOM

Open hongshuo-wang opened this issue 1 year ago • 1 comments

使用Chinese-Vicuna-lora-13b-belle-and-guanaco和llama-13b-hf运行interaction.sh，经过多轮对话后，显存直接爆掉了，请问是我哪里的配置有问题吗 GPU：4090 24G 系统：ubuntu22.04

hongshuo-wang avatar Jun 02 '23 02:06 hongshuo-wang

多轮对话，上下文会越拼接越长，显存消耗会逐渐变大，要把max_len设置好

Facico avatar Jun 29 '23 09:06 Facico