FastChat icon indicating copy to clipboard operation
FastChat copied to clipboard

Published 20 hours ago • verified publisher icon lm-sys

Fine Tuning on Low Memory GPU

Open samarthsarin opened this issue 2 years ago • 2 comments
trafficstars

Hi Team,

First of all I would like to say the model is very good in answering questions. Great work by the team. Just a suggestion if you can help with some modifications and fine tuning scripts which can help everyone to fine tune vicuna on custom datasets on low end GPUs as well like 16GB. Also, I see a major dependency on A100/H100 GPU? Can you provide some light on the possibility of fine tuning on low memory GPU as well?

Looking forward for your response. Thank you

samarthsarin avatar Apr 22 '23 18:04 samarthsarin

also hoping to support gpu acceleration on mac.

SAKURA-LBY avatar Apr 25 '23 07:04 SAKURA-LBY

#543

zhisbug avatar Apr 25 '23 08:04 zhisbug

@samarthsarin it's been a while. Did you manage to do so on the small-ram gpus?

surak avatar Oct 21 '23 16:10 surak

Closing it for now. feel free to open if you still have question.

infwinston avatar Oct 21 '23 17:10 infwinston