StableLM icon indicating copy to clipboard operation
StableLM copied to clipboard

Published 20 hours ago • verified publisher icon Stability-AI

Colab OOM

Open amrrs opened this issue 1 year ago • 2 comments

Hey THanks for the code. Ironically even the 3B model is crashing on Colab. This is after enabling 8-bit with fp16 precision.

Did it work for anyone?

amrrs avatar Apr 19 '23 17:04 amrrs

Had the same error, seems like that the CPU RAM is not enough to load the model before sending it to the GPU.

gustrd avatar Apr 19 '23 17:04 gustrd

Maybe this is a reason - https://github.com/Stability-AI/StableLM/issues/6

amrrs avatar Apr 19 '23 18:04 amrrs

Yep, that's why, and there's solutions on that thread!

mcmonkey4eva avatar Apr 24 '23 17:04 mcmonkey4eva