alpaca-lora icon indicating copy to clipboard operation
alpaca-lora copied to clipboard

Published 20 hours ago • verified publisher icon tloen

Low memory crash while loading shards

Open francis2tm opened this issue 1 year ago • 1 comments

Hello, Clearly my PC is not as juicy as yours. I'm attempting to load the model with LlamaForCausalLM.from_pretrained() but the script crashes since I don't have enough RAM (I only have 16GB). Is there anyway to load with only 16GB?

Thanks in advance

francis2tm avatar Mar 22 '23 01:03 francis2tm

Seems the lora train need 18G: https://twitter.com/nash_su/status/1637423768665718784 And I would like to know the generate.py also need 18G vRAM?

CarlDegio avatar Mar 23 '23 07:03 CarlDegio