llama.cpp icon indicating copy to clipboard operation
llama.cpp copied to clipboard
Published 1 month ago • verified publisher icon ggml-org

我想convert一个比较大的模型时报错Unable to allocate 1.96 GiB for an array with shape (128256, 8192) and data type float16如何解决

Open zhaoyuchen1128 opened this issue 1 year ago • 1 comments

What happened?

模型有141G

Name and Version

latest

What operating system are you seeing the problem on?

No response

Relevant log output

No response

zhaoyuchen1128 avatar Jun 20 '24 08:06 zhaoyuchen1128

Looks like classic OOM (out-of-memory) error, I'd advise using smaller model or more quantized model. Without much context, it is hard to tell the issue.

M-Ali-ML avatar Jun 21 '24 12:06 M-Ali-ML

This issue was closed because it has been inactive for 14 days since being marked as stale.

github-actions[bot] avatar Aug 05 '24 01:08 github-actions[bot]